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PREFACE 



This study had its origin in informal discussions that occurred in late 1^63 between 
the writer and several far-sighted and research-minded individuals In the U. S. 

Education— all professionally concerned with the improvement of foreign language teaching 
-who suggested that there was a need for a thoroughgoing survey of the preparation of 
foreign llnguage teachers in the colleges and universities of the United States. In 
partiLlar, I Lve in mind Drs. D. Lee Hamilton, Kenneth Mldenberger, A. Bruce 
Gaarder, who pointed out that such a survey had recently become much more feasible due 
to the availability of the then new MLA Foreign Languag e Proficiency Tests for Teache ^ 
and Advanced Students, designed to measure foreign language skills and teacher compet- 
7nces at an advanced level. The idea of conducting such a survey had immediate appeal 
for me, particularly if it could be done on a large enough scale and with the 
of sufficient auxiliary data to enable one to assess in depth the factors associated with 
the attainment of foreign language proficiency. Such a survey seemed also “^de to order 
to contribute toward the carrying out of one of the provisions of the National ® 

Education Act of 1958: "The Commissioner is authorized, directly or by contract, to make 

studies and surveys to determine the need for increased or improved ^ 

foreign languages..." (Section 602). As such, it might provide a model for similar nat 
ional assessment" studies at other educational levels, or indeed in other curriculum 
areas besides foreign languages. 

I therefore welcomed the opportunity to plan and conduct the study reported here. 

The gentlemen named above were of great assistance in helping me make arrangements for 
negotiating a contract with the Department of Health, Education and Welfare. A success- 
ion of coordinators in the Language Research Section of the Bureau of Research of the 
U. S. Office of Education— Drs. James Alatis, Augustus Koski, drying R, Wershow, and M 
Storm and his assistant Mr. William Higgins— have been exceedingly helpful in smoothing 
the way when it came to such matters as clearance of forms with the Bureau of the Budget, 
the obtaining of certain statistical data, and the negotiation of contract extensions 
that became necessary. To all of them I express deep gratitude. 

At an early stage of the study, several people in the New York State Education De- 
partment helped in facilitating the participation of colleges and 

state in the pilot investigation conducted in the spring of 1964. Mr. Norman D. Kurland, 
then Consultant on College Proficiency Examinations, deserves special mention. 

The Modem Language Association, with its concern for a host of matters connected 
with foreign language teaching, has given generous moral support to this investigation at 
all stages. Most of the individuals already named as being in the U. S. Office of Edu- 
cation were in fact members of the MLA who, from a professional standpoint, were acting 
in behalf of some of the general purposes of the MLA Foreign Language Program in promot- 
ing this study. After returning to the MLA central office from his post in the U. S. 
Office of Education, Dr. Mildenberger continued to evince great interest in the s u y ^ 
has been of much assistance in publicizing the preliminary results. Dr. John Hurt Fisher, 
as Secretary of the Association, has watched the progress of the study with a benign eye. 
Dr. Joseph G. Astman, and later, Mr. F. Andr6 Paquette and his assist^t Mrs. Suz^e 
Tollinger, charged with various matters concerning the development and use of the MLA 
Foreign Language Proficiency Tests, have given much helpful information and advice. 

It would have been virtually impossible to conduct the study without the cooperation 
of Educational Testing Service. Dr. Robert Solomon, Dr. Peter Loret, Miriam ryan, an 
Olive Bray helped arrange what seemed to the writer the mammoth task of administering and 
scoring tests given on tight schedules at several hundred different institutions throug - 
out the United States. I am very grateful to them. 

Thanks also are due to Dr. James Frith and Miss Claudia Wilds of the Foreign Service 
Institute, Department of State, Washington, D. C. , who managed to arr^ge, . 
obstacles, the giving of FSI language proficiency interviews to more than a hundred per- 
sons at seven different ^institutions in the eastern paft of the country. 
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on Time 20-21 1966. a mnnber of consultants were called together at Harvard to 

ex»l^ irpreliiinfr; resuJ“ of the survey and give suggestions quest- 

ions that might be answered by suitable analyses of the data. I am thankful to all th 
members of that most interesting and fruitful conference. They were. 

Dr Edward Allen, College of Education, Ohio State University 

Dr! Joseph G. Astman, Department of Foreign Languages and Literatures, Hofstra 

Dr. D^igh^BolLger, Department of Romance Languages, Harvard University 
Dr. Nelson Brooks, Modem Language Department., Yale University 
Dr Alfred Hayes, Center for Applied Linguistics, Washington, D. C. 

Dr! Wallace E. Lambert, Department of Psychology, McGill University 
Peter Loret, Educational Testing Service 
Dr. Andrea McHenry Mildenberger , Modem Language Association 
Dr! Kenneth Mildenberger, Modem Language Association 
Mr. F. Andr4 Paquette, Modem Language Association 

Dr. Jack Stein, Department of Germanic Languages, Harvard University 
Mrs. Suzanne Tollinger, Modem Language Association 

At Harvard I have received welcome support from Dean Theodore R. Sizer of the 

Graduate School* of Education, Deans Franklin Ford and John Monro of Harvard College, and 

ElUott of Radkiffe College. Dr. Dean Whitla of the Office of Tests as- 

siSed in arranging testing at Harvard. Dr. Theodore Colton of the Harvard Medical 

«5rhool was esoecially helpful and generous at all stages in giving advice concerning the 
1 ■! of the study* later. Professor William G. Cochran of the Department of 

S^?mLfhtSef tf reSv^so^rSimcult problems of data analysis and interpretation. 
The facilities*^of the Harvard Computing Center were of enormous aid, as well m the pro 
grams developed by Drs. William Cooley, Paul Lohnes, and ^““eth 
students or faculty members of the Harvard Graduate School of Education. 

Throughout the three-year period of the study, I have been fort^iate in Having the 
services of an excellent staff. As Research Assistant, Mr. (now Dr.) John L. D. Clark 
was primarily responsible for working out the procedural details of data collection for 
the study in both its first and second phases, also some of the 
Mr. ThomL M. Edwards, Research Assistant, was primarily 

details of statistical data analysis in the second, major phase of the study. Mrs. (now 
Dr ^ Fannie A. Handrick has at various times served as a Research Assistant especially 
concerned with the analysis of data from the department chairmen's questionnaires; Chap- 
ter IX of this report is based on her doctoral dissertation. Mrs. Hary aaren s 
2 a Lvoted Prolec^^ throughout nearly the whole period of the study and pe^ 

formed many seemingly thankless clerical tasks in reducing the mountains of data to man 
ageable proportion!. Miss Marjorie M. Morse, Jr., Mrs. Ellen W. Davis, Mrs. Nancy 
s!derberL and Mr. Robert Stryker gave expert attention though countless hours of cod- 
ing punching, card sorting, and running of statistical taDulations. During the final 
stages oi the study. Miss Sally Chapin cheerfully allowed hersell: to be pressed into 
semice to type the camera-ready copy of the report with its endless statistical tables. 
My regular secretary and administrative assistant, Mrs. Edith A. Alexander, was helpful 
in numerous ways. To all these people I am grateful in more ways than words can say. 

Although I was assisted by a great many people, none of them should be held respons- 
ible ^r any errors of planning, execution, or interpretation that may be noted in this 
report. I accept sole responsibility for this study and the present report. 

The report had to be written, unfortunately, at a time when I was in transition 
between Harvard University and a new post as Senior Research Psychologist at Educational 
?estiS SeS^ice, Princeton, which I took up in February of this year. I am thankful to 
my colleagues at Harvard for continuing to provide me with the facilities ^ ° 

cLplete the project, and to my colleagues at Educational S 

me time away from other duties. But I am even more appreciative to my wife, Mrs. Mary . 
Carroll, for allowing me time off from my responsibilities to her and our home in this 
rather difficult time of personal upheaval. To her I hope I can promise that wr ng 
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pressures will never again be as heavy, but she has every right not to believe my 
promises. 



John B. Carroll 



Cambridge, Massachusetts, July 31, 1967 



Soecial note: The original data of this investigation, save for information that would 

identify institutions and individuals tested, will be archived at the ERIC Clearinghouse 
on Foreign Languages, Modem Language Association, 62 Fifth Avenue, New York, N. Y. 

10016. They will thus be available to qualified investigators to use in further analyses. 



Chapter 1 



INTRODUCTION TO THE STUDY 



1. Purposes 



This study had two major foci: 

(1) the study of foreign language teaching in the United States in so far 
as it affects students who "major" or "concentrate" in foreign languages at the 
collegiate level; and 

(2) the study of the collegiate preparation of foreign language teachers. 

In the conduct of the study, it was first necessary to make an assessment of 
the levels of foreign language competence attained by foreign language majors near 
the time of their graduation from college, and then to examine the student character- 
istics and instructional program variables associated with the various levels of 
measured competence. In this way it was hoped that useful conclusions could be drawn 
concerning the whole state of foreign language teaching in America and that certain 
suggestions would emerge as to ways in which instructional programs might be improved. 

In the case of students preparing to become teachers, attention was directed, in 
addition, to the degree to which these individuals had been able to attain not only 
the basic linguistic skills in their respective languages but also certain skills and 
knowledges pertaining specifically to their preparation as prospective foreign language 
teachers > This part of the study was therefore designed to produce useful Information 
about the conduct of foreign language teacher preparation programs. 

Ever since the establishment of its Foreign Language Program in 1953, the Modern 
Language Association of America has evinced much concern for the adec^uate prepara- 
tion and certification of foreign language teachers (Axelrod, 1966; Paquette, 1966). 
With the support of funds from the U. S. Office of Education and with the assistance 
of Ed^lcational Testing Service, the MLA mounted a major project in the late 1950 *s 
to develop a series of proficiency tests in foreign language skills and teacher prepa- 
ration knowledges in five languages: French, German, Italian, Russian, and Spanish 

(the five languages designated as "commonly taught" modern languages by the U. S. 

Office of Education). By 1962, this project eventuated in the MLA Proficiency Tests, 
for Advanced Students , constructed in two forms in each language (Starr, 1961a, 1961b, 
1962; Educational Testing Service, 1964a). Because these tests were carefully con- 
structed by foreign language experts with the help of measurement specialists and laid 
at least as much stress on listening and speaking skills as they did on skills in read- 
ing and writing, they have come to be widely accepted in the foreign language teaching 
profession as valid and useful measures. Many states have adopted them for use in the 
certification of public school foreign language teachers, and they have been employed 
also by many college and university foreign language departments for testing the 
attainments of students at various stages of advanced training. They have received 
extensive use at NDEA Institutes for Foreign Language Teachers to assess the effects of 
institute training, and preliminary norms have been derived from the results. Recently, 
the administration of the tests has been undertaken by Educational Testing Service as 
one of its regular national testing programs. 

The immediate impetus for the present study was the need for carefully collected 
normative information felt by states and other jurisdictions using the MLA tests for 
teacher certification. But as early as 1963, a conference, convened to consider prob- 
lems of the preparation of college foreign language teachers, had recommended that 
comprehensive studies be made of the attainments of both undergraduate and graduate 
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students as measwred by the MLA tests. To quote from the conference report! 

'•To enable these tests [the MLA Prdficlency Teats for Teachers and Advanced 
Students] to produce the results of which they appear capable, the Conference 
recommends that a project be undertaken, with government or foundation support, 
to arrange for and finance the experimental administration of the I'iLA Tests in 
the four language skills, on the largest possible scale, to college majors in 
the relevant languages at a time near graduation and to graduate students toward 
the end of one or two years' residence. Definitely superior to the Graduate 
Record Examination In the measurement of language skills, these tests, if ad- 
ministered at the end of the Undergraduate Program and again during the Graduate 
Program, would enable language departments to measure the achievement of their 
students according to a recognized scale. The results of this testing would 
determine each student's need for additional language training in graduate school. 

If the tests were to be used as partial demonstration of a student s readiness 
for teaching, the Conference felt strongly that they should be taken again when 
the student proposes to begin his teaching career: the proficiency must be measvred 
idien it is to be used and not at some time in the past" (MacAlllster, 1964, p.3.'). 

Limitations of funds and other resources made it infeasible for this study to use 
the MLA tests in an investigation 6 f foreign language attainments at th^- ^aduate, 
school level, but the study has, in effect, attempted to carry out the QmHtence 
recomendations with respect to testing at the undergraduate level. In 
ploying not only the four skill tests but also the three professional preparation tests 
for students intending to teach, it went beyond the Conference recommendations. 

An Important aspect of the present study was its attempt to calibrate the scores 
on the MLA Proficiency Teste in terms of quasi-absolute. Inherently meaningful^ standards. 
Sere were, to be sure, certain tentative calibrations available from a study by Myers 
and Mtelton (1964), but ordinarily, scores on those tests could be Interpreted only in 
relative terms, i.e., with reference to the performance of known groups such as samples 
of forelgh language teachers attending NDEA Institutes. But percentile ranks and stan- 
dard sc^es fall to Indicate In any meaningful way the absolute amount of competence 
the tested individuals possess. Except to the extent that one can guess at the range 
of competence possessed by a reference group, a percentile rank cannot tell, for 
exa^L how sLcessful the individual would be In communicating with a native speaker 
STSe language or in comprehending the substance of printed materials in the l^guage. 
Therefore,*a substudy was designed to ascertain correspondences between MLA Proficiency 
Test scores and the ^absolute proficiency ratings" rendered by expert^tearas from the 
Foreign Service Institute of the U. S. Department of State (Rice, 1959). These Matter 
ratings are couched in terms of meaningful levels of proficiency and are accepted in 
U. S. Government circles as directly Interpretable assessments of foreign language 

In the course of conducting a large normative study of the foreign language 
attainments of college foreign language majors, it was realized that much valu^le sub- 
sidiary information could be collected and analyzed with regard to such questions as 

the following; 

(a) Who are the foreign language majors? That is, what are their backgrounds 

and: Characteristics? ^ « 

(b) What are their motivations for foreign language study? 4 

(c) What kinds of exposure to foreign language study have they had? What 
fluences have they had outside of their regular courses ot study in foreign languages? 
To what degree are these Influences associated with degree of success in foreign 

language s^udy observed in foreign language major elections and 

attainments in different types of institutions or in different ° 

(e) Can different types of collegiate foreign language programs be identified, 
and If 80, are these types associated with different levels of foreign language attain- 
ments in the students? 
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As the principal investigator has observed (Carroll^ 1963^^ thiS9|?e '.is still 

a dearth of reliable research in foreign language teaching. Alt^ugh the present 
study could not, by its nature, investigate foreign language learning processes and 
instructional procedures at close range, it was hoped that some of the information it 
was to collect would eventually serve as useful background for such research. 



2. Overview of the Study 



This study was conducted in two phases. The first phase started officially on 
March 15, 1964 and consisted of a pilot study in which the MLA Proficiency Tests and 
various other instruments were administered in the spring of 1^4 to 667 individuals 
at 44 Institutions of higher learning in the state of Hew York:f' The purpose of this 
"New York pilot study” was primarily to try out procedures of gathering data and only 
secondarily to obtain substantive results for the sample of individuals tested. Al- 
though the study was intended to obtain data from 100% of the senior foreign language 
majors at 100% of the institutions of higher learning in the state of New York, the 
institutional response rate was only 70.1% and the overall student response rate with- 
in participating institutions was only 76.2%. As result of experience gained in the 
pilot study, certain modifications were made in the procedures for gathering data 
with a view to obtaining higher response rates in the second phase of the study. In 
addition, tentative norms were established for the MLA Proficiency Tests on the basis 
of the sample obtained in the pilot study. A final report on this phase of the study 
was rendered on January 31, 1965 (Carroll, 1965); this report did not contain, however, 
detailed analyses of data obtained. Appendix A of the present report presents a summary 
of the principal results of pilot study data analyses that have been made subsequently. 
These data analyses are not included in the main body of this report because they are 
not considered as representative and significant as the analyses that have been made of 
the data obtained in the second phase of the study. 

Planning for the second phase of the study began in the fall of 1964, although this 
phase did not start officially until December 1 of that year. Instruments and pro- 
cedures were revised in the light of experience in the New York pilot study. A sampling 
design was devised on the assumption that it would be possible to obtain a 100% stu- 
dent response rate within a stratified pptimal sample of 192 institutions of higher 
learning in the whole IMited States such that a probability sample of 50% of all foreign 
language majors in the country would be tested. Contacts x/ith institutions were made 
beginning on December 10, 1964; because of a disappointing institutional response rate 
in this first sampling„a further sample of 102 institutions was chosen and solicited. 

In the end, however, only 2,875 students were tested in the spring of 1965 (rather 
than the approximately 5,000 desired), and there were only 203 institutions that parti- 
cipated. The present report is mainly concerned with describing and analyzing the 
findings of this second, nationwide sampling study. 

The substudy of MLA Proficiency Test equivalences with FSl absolute proficiency 
tests was done in the spring and summer of 1965, with the use of selected foreign lan- 
guage teachers enrolled in NDEA institutes. 

The work of makihg the major analyses of the data collected in both the New York 
pilot study and the nationwide study was accomplished in the academic year 1965-66 » 

In June, 1966, the principal results obtained up to that time were presented at a 



*The pilot study was also to include all foreign language majors graduating from 
Harvard University and Radcliffe College in June, 1964. However, only 4 out of 
approximately 50 students volunteered to take the tests. 
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two-day invitational conference of consultants. These consultants, among whom were 
found both authorities on foreign language teaching and experts on educational research, 
made numerous suggestions as to further analyses that should be undertaken; these 
further analyses were made during the academic year 1986—67 , which also saw the com- 
pletion of the present report. One of the major concerns of analyses conducted during 
the academic year 1966-67 was to estimate the degree of sampling bias inherent in the 
results of the nationwide study; it was only during that year that it became possible 
to check enrollment figures against tabulations made by the statistical services of 
the U. S. Office of Education. 
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Chapter II 



MEASUREMENTS AND PROCEDURES 



1. Instruments 



MLA Foreign Language Proficiency Tests for Teachers and Advanc ed Students (Po.rm A) 

From the start of this project, there was never any doubt that the instrument 
of choice for the measurement of foreign language competencies at an advanced level 
was the series known as the MLA Foreign Language Proficiency Tests for Teachers and 
Advanced Students, constructed in five languages and in two alternate forms in an 
extensive three-year project of the Modem Language Association, under the general 
direction of Wilmarth H. Starr and with the support of funds provided by the U. S. 

Office of Education under NDEA Title VI. 

Originally conceived as "Qualifications Tests for Secondary School Teachers of 
Foreign Languages," the MLA tests gradually evolved into a wide&range, general battery 
of tests that could be used for measuring both basic foreign language skills and 
certain professional teaching competencies. As Starr (1962) recounts their history, 
n significant Impetus came from a statement of "Qualifications for Secondary School 
Teachers of Modem Foreign Languages" formulated in 1955 by the Steering Committee of 
the Foreign Language Program of the MLA. These qualifications established three 
general levels of proficiency (Minimal, Good, and Superior) for seven areas of language 
teaching competencies: (1) aural understanding, (2) speaking, (3) reading, (A) writing, 
(5) language analysis, (6) culture, and (7) professional preparation. It became 
recognised, however that "the statement of desiderata, no matter how strongly represent*< 
atlve of a consensus of the profession, would not be’ as effective as the situation de- 
manded until nationally standardized tests could be developed that would Implement the 
description of competencies." In the spring of 1959 the MLA made a contract with the 
U. S. Office of Education to develop such a series of tests in the five languages 
designated as "commonly taught", namely, French, German, Italian, Russian, and Spanish, 
and Starr, then Head of the Department of Foreign Languages and Classics at the Uhl- 
versity of Maine and later Head of the All-Uriiversity Department of Romance and SiAvic 
Languages and Literatures at New York University, was named Project Director. The 
actual work of test constmction was carried out by 31 committees (one in each language 
in each of the first six competence areas, plus one all-language committee in the pro- 
fessional preparation area) composed of foreign language teachers from both secondary 
school and college levels. These committees were assisted by test construction experts 
from Educational Testing Service, which also conducted most of the work of administering 
the tests in field studies, and performing statistical analyses of the results. The 
tests went through preliminary editions before being refined, through item analyses 
and test critiques, into the forms that are now available from Educational Testing 
Seriice. The final standardization runs were conducted in 1961 with testing populations 
from 68 Summer and 7 Acadiemic-Year NDEA Institutes for foreign language teachers. An 
authoritative source of Information about the tests is a booklet published by ETS 
(196Aa) , A description of the MLA Foreign Language Proficiency Tests for Tee chars and 
Advanced Students c containing the (slightly revised) qualifications statemet*ts for 
Superior, Good, and Minimal levels in each area of competence, sample questions, and 
statistical infoemation on reliability, means and standard deviations, and inter- 
correlations for the NDEA Institute norm population. Percentile norms are published 
by ETS in a separate leaflet (ETS, 196Ab) , and more detailed information is available 
on item statistics, etc., in various publications of restricted circulation. Myers 
and Mslton (196A) conducted a study attempting to set up equivalences between MLA 
proficiency test scores and MLA-defined competence levels (Superior, Good, Minimal, 
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«nd Unsatls factory) . 

In an address given at the 1960 convention of the MLA, Starr (1961i^ made three 
major points about the tests. First, he said that t^ey ”are pitched rather higher than 
^ well known foreign language tests of the College . 

Advanced Placement Test batteries.” Second, "they are pjE^ifiiSSSZ. tests and 
reflect the general and growing dissatisfaction with 

which looks i?on teacher qualification quantitatively under the assumptlra that hours 
of contact with subject matter in the classroom and credits gained thereby insure the 
necessary qualification to teach.” And third, "they are pioneering new frwtlers. 

Aa far as he knew, "it is the first time the members of an acad^lc subject^atter 
area have tried to identify and standardize their own competencies on a tuitApnal scale. 

The tests have been remarkably well received not only do 

teaching profession itself but also by many state and local 

with the certification or selection of foreign language teachers. At this writing, 
at least 16 states use them in this way, and the MLA has established 
by P, Andrd PaquettO, to give advice and assistance on tke use of the ™ 

also has a continuing contract with the U. S. Office of 

the teats and to keep them current through the creation of further alternate forms It 
the cooperation of Educational Testing Service. The latter agency now conducts a 
regular national MLA proficiency testing program 5 times a year. In the 
10^00 persons were tested in various programs, national and others. In addition, e 
tests continue to be used in NDEA Inatltutes and in institutional testing projpms. 

The testa are not, of course, perfect, but the evaluations f 
Buros’s Sixth Mental Measurements Yearbook (1965) were in general highly 
The MLA has reprinted these evaluations, together with a large n^ber of . 

evaluations it itself collected from foreign language teachers, in a document (lollinger 

and Paquette, 1966) for restricted circulation. 

The series consists of a battery of seven tests in each of five ^®®®®®®®® ““ 

Geman. Itallm, Roaalaa, and Spanlah. The flrat four te«a In ea«^ batte^ co TO r th^ 

.kill arena— Uatenlng, speaking, reading, SvlllLtlon 

areas of competence Important In language 

and culture, and professional preparation. Although following a comnon pattern and 
teat specifications, the queatlona In the first si* tests are different 
language; they are In no sense translations of each other. The sewnth test. Profess 
lonal Preparation, Is entirely In English and Is the same for all languages. 

The seven tests In each battery are described briefly below, with approximate 
administration times, number of Items, and maximum converted scorM for Pora A Ithe 
form used In the present study) . Descriptions have beep freely adapted from the booklet 

issued by ETS (1964s) . 

1. Hetenlne Comprehension Test (20 minutes, 36 ltema,.m«i_cogyerted score - 56) 

The student listens to a tape recording > e 

dialogue, a radlo?*roadcaat, and a three-part conversation. For all but tbe last part, 

the ex^nee selects, from among four choices printed ri or 

foreign language) g the responses most likely to be made In the at on his 

tS^biat aniwria to queatlona put to him. Por the last part, the examinee on his 

Mtwer sheet whether each of a number of statements about the 
heard Is true or false. Questions are designed to test phonetic (or phoney ) 
natlM, command of Idiomatic expreaslona, voMbulary, and 

aatlon use of the language. Comprehenalon of main facts. Ideas, and Important 
is evaluated# 

While the main object of this teat la to measure 
pends to some extent upon reading comprehension because the alternatives to the multi 

pie choice questions are printed in the foreign language# 
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•) qneAkltitt test (15 minutes, 105 items, max. converted score - 125). 

^Is tLt has three parts. The student records all his answers on tape for later 

evaluation by expert scorers approved by MLA and ETS. 

In Part A the student repeats 15 sentences spoken on the tape by a 
Tn Part B he is oresented with short printed texts which he must read aloyd into the 

parts he is scored for the "rightness" or ' Wgnew" 
renroduction of certain specific sounds or intonation patterns e In addltiw, 
t 5ert rSe ?rs“red L the .derail quality of his reading on a five-point scale. 

Tn Part C. the Is presented with a single picture, a pictorial sequence, 

and a situation shown plctorlally (with directions printed In English) • In each eas^ 

S la JrdesrSbrthrplctire or to react orally to the sttuatlon P«s»ted. On 

five-point scales, he is rated on Vocabulary, Pronunciation, Structure, and Pluen^. 

3. gaadlne test (40 minutes, SO items, max. oonverted score - 70) 

Tn Part A. the Student is to select the appropriate lexical, idiomatic, or 
structure? coJ^;i2U for each of 15 incomplete statements. Part B conslws of ^ral 
ahMtnassages followed by questions (30 In all) on the meaning of words and phrases 
as ^llT~ tS content of the selections. Part C contain 5 

poetic excerpts. All materlals—readlng passages and questlons—are In the fore gn 
language e 

L Mrltlne test (45 minutes, 60 items, maxe converted score ■ 80) e^ _ ^ 

Ml itansTn this test are o^ the free response type, each to be scored 
tironff bv exoert scorers e Part A consists of short texts in which there are, in all, 

30 blanL representing omitted words that are to be filled in from the context, 
sLdeS LwlieL on a separate answer sheet, a single word that is appropriate both 
in Lading Ld form. (In the case of the Russian /he b^e fom of each word is 

supplied, and the student is to supply the appropriate inflected form.) Part ® . 

"Interlinear exercise" consisting of texts "poorly written (as If, say, by a st^ent) 
cOTtSninc In all. 30 grammatical, lexical, or idiomatic errors that are to be fomd 
^ corierted by the «Se. Th4 examinee la scored right for each error properly 
M«^ted »d “^each error Improperly corrected or unnoticed. 

given for performance on parts of the teat containing no planned errors. 
teat differs slightly In that the errors In the first text are Identified by under 

lining.) 

i; Aiqni-iAfl linsuistics test (40 minutes, 55 items, max. converted score - 75). ^ ^ 

’ me items cover matters of pronunciation and phonetics, orthography, morphology and 
Anfrenerel! historical, and comparative linguistics. Questions in this test 
are couched in English except for cited examples in the forel^ Imguage. T^e Senera 
emphasis is on those differences between English and the foreign language that are 

importance in teaching. 

t «,rfH,,tlo.i and culture test (30 minutes, 60 Items, max. converted score - 80^ 

pSm ^U i ple-cholce quest i o ns In English, this test endeavors to measure the _ 
knowle*^of civilization »d culture (as distinct from knowledge of the l“8^e) needed 
w a tMcher to Introduce his students IntelUgently to that area of the world 
the laneuaae is spoken e The term "culture” is taken in the broad sense, embracing such 
vt^on^^AreA8 as (geography, history, the arts, literature, and social institutions e 
Sm^tSH context, quLJlons are designed to test recall of basic facts term^ 
of important ideas, themes, and institutions; understanding of the com- 
Mexity and variety of the culture; appreciation of historical and cultural meaning o 
terms •^appreciation of cross-cultural differences; and understanding of relationships 
not oily^Lnong basic facts and terms, but among ideas, themes, and institutions as well. 

7 PvofAAAlonal nreoAration test (45 minutes, 65 items, max. converted score - ®5). 

This test is^the same for ah languages. It consists of multiple-choice questions 
j .f j «•Aa^ not onlv knowledge of teaching methods that are common to different 

i^S^es Mrt c-ehMi? 
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developments In the field of language testing. Questions deal with such areas as 
objectives, teaching techniques, professional development, and evaluation. 

Torm A of the MLA series was selected for the present study because, at the time the 
testing was done. Form B was reserved for teacher certification testing and Form C (now 
available for all languages except Italian) was still under development. 



The Modem Language Aptitude Test . Short Form 

An Important student characteristic that needed to be taken account of In this 
survey Is the student’s basic aptitude for learning a foreign language. 
reviewed by Carroll (1963) has suggested that foreign languap aptitude Is a relatively 
invariant characteristic of the Individual (differing among Individuals, ^ 

Is associated with the ease and rapidity with which the Indl^dual can leam a 8 

language. A measure of foreign language aptitude was desired for this study In ord^ 
not only to have a variable that would permit the control of various experimental com- 
parisons but also to study further the relations between language aptitude and the 
attainment of competence in foreign languages under various conditions. 

The Modem Lanauaee Antltude Te_^_ (Carroll and Sapon, 1958) was selected for this 
purpose because It was considered to be the best test available ^ 

scribed In the accompanying test manual. It had been extensively validated ^® ^® 
diction of success In both traditional and audlolingually-orlented foreign lang g 
Instmctlon (see also Carroll, 1962; Carroll et al. , 1966). 

The complete test, containing five parts, requires an administration time of about 
60 minutes and necessitates the use of a tape recording. Parts 3, A, ^d 5, hwever, 
constitute a "short form" which altogether requires only about 30 minutes “d does not 

entail the use of a tape recording, consisting entirely of the 

reliability and validity are satisfactory although probably not as as ^“® 

complete tLt. To minimize testing time the short form was selected for JJ®® ^® 
present study. A special answer sheet was prepared so that the test could be ^ 

the SCRIBE scoring machine at Educational Testing Service, along with the MLA proflcl 

ency test answer sheets. 

Part 3 of the MLAT, entitled "Spelling Clues," requires the subject to recognize 
words "spelled approximately as they are pronounced," such as 1^ (love) md ems t 
Earnest) and choose a synonym for each word from five ch^cea offered C affectl || and 
sincere are the correct responses for these examples). The test contains 50 1 , 

for which a time-llmlt of 5 minutes Is allowed, and Is thus highly speeded. Accor ng 
to the test manual, "scores on this part depend to some extent on ^^® f ^J®"^ ® 

English vocabulary knowledge"; however, it "also measures the same kind of 

symbol association ability as measured by Part II, Phonetic Script, ® f 

extent." [Part II Is a test that must be administered by means of a 

Tills test? the subject learns to associate particular English sounds with symbols 

In a special phonemic transcription.] 

Part A of the MLAT, entitled "Words In Sentences," requires the subject to select 
elements of sentences that correspond In grammatical 

Other, "key" sentences. It can be characterized as a test of 8i^a®n»a“cal an 1 g 
Acwrdlng to the test manual, "this part is thought to measure sensitivity gr^- 

matical Structure, and may be expected to have particular ’"®^^^®S®® minutes are 

ability to handle the grammatical aspects of a foreign language. J^^^®®"^ speeded, 

allowed for the A5 Items of the test: for most students, the test Is not highly speeded. 

Part 5. entitled "Paired Associates," requires the subject to J'^® 

meanlnas of 2A nonsense words labeled as "Kurdish" (they are not really Kurdish) » ^ 
Xief are this memorization, after which there Is a 4-minWte. non-speeded 

multiple-choice test on the memorization e Maximum score Is 2A. 
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Questlonnalre for Foreign Language Majors 

It was deemed desirable to collect from each student in the testing program a wide 
variety of information concerning his background • interests t and attitudes^ in the study 
of foreign languages (particularly, the language in which he was "majoring ). Such 
information was to be studied in relation to his test scores* 

The Questionnaire for Foreign Language ftajors used in the national testing program 
was a revision and adaptation of a Questionnaire for Students Participating in the Pilot 
Proeram that had been developed for the New York pilot study. Refinement consisted 
primarily of editing phraseology and converting certain questions from a free-response 
to a multiple-choice form. The final form of the questionnaire is to be found in 
Appendix B. 

Sssentlally, it has three parts. The first page asks for the student's name, the 
permanent address to which the test results could be mailed, sex and age, expected date 
of graduation and institution, language in \dilch he would be tested and data on any 
other languages Included in his official "major subject," years of study had in other 
languages, and whether he had ever taken the MLA Proficiency Tests before. The second 
part is contained on pages 2 and 3 and asks for a detailed (but student-coded) history 
of the courses he has taken in his major language. Including information on courses at 
"grade school level," high school, and college, the duration of the courses, class hours 
per week, size of class, teacher's classroom language, student’s classroom language, use 
of language laboratory, quality of teacher's pronunciation, use of native informants, 
whether the final examinations Involved listening and speaking components, and the 
student's final grade. The third part, on page 4, explored the reasons the student 
majored in a foreign language, his possible intentions to teach and if so at what levcils, 
his ratings of the relative Importance of listening, speaking, reading, and writing 
gltillg to him, plans to qualify for a teaching certificate in foreign languages. Informal 
courses taken in the major language, courses taken in Latin or Greek, travel abroad in 
the country where the major language is spoken, parents' use of the major language at 
home, other opportunities to use the language, and the amount of reading experience in 

the major language. 

Information obtained with this questionnaire was mainly of an objective nature; the 
ways in which the answers were coded and summarized will be set forth in later chapters 

of this report. 

Questionnaire for Chairmen of Foreign Language Departments 

Ifi order to gather data on collegiate instructional and teacher training programs 
that might be found relevant to the interpretation of the results of the proficiency 
testing, a Questionnaire for Chairmen of Foreign Language Dep artments was used (Appendix 
C). Except ** "^nnr change (correcting German Review to Gerniahic Review) this was 
identical to the questionnaire of the same name used In the New York pilot study, since 
the pilot study had not disclosed any other suggestions for revising it. This form 
asked for information on: 

Languages in which courses are given in the department 

Time at which students normally elect majors 

Guidance given to students with regard to election of foreign language majors 

Requirements for acceptance of students as foreign language majors 

Begree requirements for foreign language majors 

Degree programs offered beyond the A.B. 

Number of majors enrolled In the department, in each language 

Relevant professional periodical holdings in the institutional or departmental 

library 

Types of foreign language courses offered , v j 

Special activities offered in the foreign language program (e.g. study abroad, 
language houses, language clubs, etc.) 

Policies in teaching the basic skill courses 
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TJ8« of language laboratory 

gaount of use of Engllah In foreign language courses 

A amount of the Information obtained In this questionnaire was of a 

te ?e h«.d-cod.d and s»».rlzed In a aerlea of atatlaolcal 

varlidilesa 



2. Tha eatqbllflhment of tentative MI A nroficlencv test 

with FSI (For elttn Service Institute) 
flhftftlute profic iency ratings 

nna of the Important ohjectlvea of the atudy waa to eatabllah norma on the 

:r^5sM-ss rusrrrax* • 

language* 

— - a. ^ V.. u»A«.a Atut Melton rl964) attempted to prodisce meaningful ranges of 
^ Teste in teLs of tie adjectival standards (Super- 

tor“ccX^ ””"*^ I Unsatisfactory) that had been set by the Itodern ^"“|® 

in it. ttuSs ^ 

riss 

Language Institute g /T-io^o*» 4 no <«nMklnee etc a) presumably measured by the 

‘t^t^ncsrrfin^ Tcrf^irSafre 

faculty groups rather than by tMt scores ,*at least, not those 

***^ TTf rnA*P™flSm^ TmM allnlswred at the conclusion of the Instruct- 
SnI;%rL »: «hen the test scores became available, they were 

compared with the ratings in the following way; 

"The procedure used for deterging the :^tTealh"^itJ“woSd 

distributions Into four groups In «»«* ® *^®5 *'*® L cateaory. For example, 21ft 

K^r.i!..sss?surr.rfi»rr4SLr»i, — •«“ «• 



reported 



II 



They further comment; 

,u-7s.-s:.!as;-as£j s: ~ 
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perfect (and should not .he expected to be perfect) » (Actually» for skills tests the 
correlations between corresponding test scores and ratings were as follows: French. 

•63 to .72; German. 167 to .76; Russian. .49 to .68; Spanl,sh. .62 to .72. For the three 
teacher preparation tests the correlations were generally lower: French. »36 to *55; 

German. .29 to .60; Russian. .19 to *36; Spanish. «30 to • 43- ^Author.Vs. .note . 3 Hence. It 
cannot be said that a person whose test score Is In the Superior range would necessarily 
be rated Superior In that skill by an Institute staff. Furthermore., neither the score 
nor the rating was designed to be a predictor of the other; both the test score and the 
rating were designed to be estimates of teaching competence." Cthls statement. It 
should be noted. Is debatable; as Aar as the skill aress are concerned.. one would think 
that the test scores and the ratings were designed to measure aspects of foreign lang- 
uage competence . —Author * s note . I 

■ Above all. however, one could question the validity of the ratings., since they were 
based on general Impressions of performance rather than upon standardized procedures of 
assessment. It therefore seemed that a new attempt should be made to establish meaning- 
ful equivalences to absolute standards, at least for the four skill areas. 

Since the late 1950* s. the Foreign Service Institute of the U. S« Department of 
State has used a standardized Interview procedure to render what are called "absolute 
language proficiency ratings" on two scales: S (Speaking Proficiency), and R (Reading 

Proficiency). The procedure, as described by Rice (1959). Is applicable to any lang- 
uage. and is conducted by trained teams composed of a linguist and a native speaker of 
the language. Interviews are conducted on an Individual basis and take about 30 to 45 
minutes. The ratings are rendered on essentially a slx-polnt scale, from 0 to 5. ex- 
cept that all the ratings except S^Vaay be modified by a plus ('I). Indicating that pro- 
ficiency substantially exceeds the m|£nimum requirements for the level involved but 
falls short of those for the next higher Ipvel" (Foreign Service Institute. 1963). 

(Thus, the scale may be regarded as an |t-polnt scale for statistical purposes; l.e.. 

0. 04-. 1. 14-. etc.) Five points on the scale (1. 2. 3. 4. 5) are accurately . described 
both in a short definition (see bottom of Table 2.2) and an amplified statement avail- 
able In a Foreign Service Institute Circular (Foreign Service Institute. 1963). Al- 
though the two scales are named Speaking and Reading, the descriptions also include some 
reference to listening skills (for the Speaking Proficiency scale) and to writing skills 
(for the Reading Proficiency scale) • The meanings of the scale points are widely ac- 
cepted and apparently well understood in V. S. government circles. 

It was therefore decided to attempt to establish equivalences between scores on the 
skills tests of the MLA Proficiency Tests and the FSI absolute proficiency ratings, at 
least for French. German. Russian, and Spanish. 

The original plans for this equivalency study called for the obtaining of 50 cases 
In each language, using students already tested in the national study reported here. 
Accordingly, arrangements x«ere made to send FSI interview teams to selected! institutions 
In Eastern D. S. already participating in the study. But the project staff failed to 
reckon on the fact that the regular MLA Proficiency Testing program had already been a 
sufficient imposition on the time and patience of both the staffs and students at those 
participating institutions. There was only one institution (Queens College) where it 
became possible to arrange for FSI testing. ®ad even here, only 6 cases, all In Spanish, 
were obtained (In May. 1965). 

To obtain the necessary cases, the project staff considered the possibility of ob- 
taining cooperation from selected NDEA Summer Institutes for Foreign Language Teachers 
where the MLA Proficiency Tests would be given both at the outset of the program and Its 
conclusion. If FSI te^ms could be sent to such institutes and give their interviews to 
selected students. It would be easy to compare the FSI ratings with both pretest and 
posttest MLA test scores and from such data to establish equivalences s As a matter of 
fact, through fortunate circumstances, an FSI team that had attempted but failed to 
arrange for testing of senior majors at Indiana University (one of the institutions 
participating in the study) was nevertheless able to arrange for testing, in May 1965. 
of a group of 24 Russian teadiers In an Academic Year NDEA Institute at that Institu- 
tion. 
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Accordingly • on June 24, 1965, a letter was addressed to the directors of ten NDEA 
Sumner institutes asking whether they would permit small numbers of their students to be 
tested by FSI tenims. Each institute was given a quota of 20 students, aid the institute 
directors were told that the students should represent "the broadest possible spectrum 
of ability in the language in which they will be tested.” They were asked to select for 
each language "four of your ablest students, four of your *good* students, four of your 
average students, J^our of your below average students, and four of your least proficient 
students.** As it turned out, it became possible to do FSI testing at five institutes! 

40 cases were obtained in French at Rutgers and the University of Pittsburgh; 30 cases 
in Spanish at Rutgers and Gannon; and 39 cases in German at Princeton and Hofstra. 
Together with the 6 student cases in Spanish obtained at Queens and the 24 Russian cases 
at the Indiana Uhiversity Academic Year Institute, this made a total of 128 cases. 

Table 2.1 shows the nisaber of cases in each language and the range of FSI ratings 
represented. 



Table 2.1 





Cases Tested by FSI 


and Ranges of 


FSI Ratings 






Ranges of FSI Ratings 


Language 


N 


S ratings 


R ratings 


French 


40 


1 to 4 


2 to 4+ 


German 


39 


1+ to 5 


1 to 5 


Russian 


24 


1 to 4 


1 to 3+ 


Spanish 


36 


1+ to 4 


1+ to 4 



Total 128 



Arrangements were made with Educational Testing Service to retrieve and transmit to the 
project staff the MLA Proficiency Test scores for all persons involved in this study. 
The MLA Proficiency Test scores used in subsequent analyses were the **post-institute 
scores, i.e., those resulting from tests administered toward the end of the institute 
period (whether it was an Academic Year Institute or a Summer Institute). For the six 
Spanish cases derived from the college senior testing, the MLA Proficiency Test scores 
were those obtained in that testing.* There were a few cases where for some reason the 



*The six Queens College senior cases in Spanish were not used in the analyses, how- 
ever, because the MLA Speaking teat scores and FSI ratings were clearly off the line of 
equivalence for the 30 NOEA cases. The Queens College seniors* MLA Speaking Scores were 
all QiMCh lofWer than one would expect them to be from the FSI ratings they received and In 
fact showed no relationship with the FSI ratings. Apparently the scoring standards used 
in judging the tapes were different for the MDEA Institute and national study cases. 

This finding is disturbing and casts doubt on all equivalences for MLA Speaking scores 
indeed upon the interpretation of those scores. The Queens College cases could, 
nevertheless, have been pooled with the NDEA Institute cases in the derivation of equi- 
valents for Listening, Reading, and Writing scores; the correlations between correspond- 
ing tests and FSI ratings when all 36 cases were used were actually higher than for the 
NDEA Institute cases alone. However, to preserve uniformity in the data it was decided 
not to include the Queens , College cases in the equivalency analyses for any scores. 
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MLA Proficiency Test scores were not available. 



Table 2.2 shows a statistical sunmaty of the basic data obtained In this FSI^MLA 
equivalency study. 

An isssedlete question is whether »hon®re«uUr 

representative of the groups tested In reoort * Statistical comparisons with 

are to be presented In Chapters III to IX of this report, ^tatist^^ 

results from the national study (Table 4.1 . mpter Ws^^rt^^^ ^8 

group Is significantly superior (at ^ significantly superior 

and Speaking, but not In Read g iA j Ciq^Aklne* the Russian group is repre— 

(at tL IX level) ‘‘>%““onal sroup «ly (!% level) ii 

speaking, and significantly ^“T«l«,J“g^Clx“ptTSlk«g! lignlfl- 

sentatiye of the natW gro^ in »TJ/^il%:“:^ptrlsons (G4man. Speaking; 

cantly inferior (1% level). There were y «roun were significantly (IS level) 
®‘■“iS^rafl:S^f^e v«ir«r“%Si naU»My!"*On t?e whole, the TSI equl- 
TOlency groups are not markedly unrepresentative of the national stu y groups. 

The mean levels__of FSI ratings 

the neighborhood of 'minimal profess o*'® of 2^ '* limited working proficiency.” 

: r.u’jssv-s: 

flclency. 

correlations between FSI ratings 

1„ the Russian group. They ^ wlth^he^^^ ^ 

corresponding are Sgarded as corresponding to both Listening 

the present study, FSI r® _ 8 ^ Reading ratings are regarded as corre- 

and Speaking seores on the MLA tests, and testa.) The correlations between 

spending to both Reading and Writing scores on the ^ f^ Prench to .90 for 

the two FSI ratings, S and R. are qjjite high ranging evidence in 

Russian. Unfortunately, save Jpg, sneaking ratings are more highly correlated 

the FSI-MLA correlations to suggest that m Spea^g^ra^^^^^ 

with MLA Listening and vlth MLA Reading and Writing scores than 

that FSI Reading ratings are more correiatea wxtn ^ on an a oriori basis, it 

Tatte^^lots of cofresSu^^^ FSI Ratings and MLA scores revealed no evidence of emrwl- 
linearity of relationship. 

Equivalences were^stabllsted b^the "equal s^^ eq^tlons 

same standard score. (A t'lS bv ?hl sUd«nevi!t?»!) Thus. If X 

$f^"F^l“«Lr^d X ““hrcorrespondlng score on an MLA test, the equal standard 
scores fre «sj^!uvely the left and right sides of the equation 



X - X ^ 



Y - Y 



This equation cm, of course, be solved explicitly either for X or for Y. Solving 
for Y, we have 

Y - (Y - X) + ^ X 



a + bx. 
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Table 2.2 

Basic Data for Equivalency of FSI "Absolute; Proficiency Ratings" 
and Scores on MLA Proficiency Tests 
for Teachers and Advanced Students 





FRENCH 


GERMAN 


RUSSIAN 


SPANISH 


Noe tested 


39 




39 




19 






30* 


Test 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S.D. 


Mean 


S .D. 


MLA List. 


47.38 


6.07 


45.62 


7.93 


41.84 


5.48 


44.57 


5.71 


" Speak. 


82.97 


9.84 


97.90 


19.83 


85.00 


10.92 


84.87 


9.49 


" Read. 


50.41 


7.82 


51.59 


11.03 


33.16 


8.96 


43.77 


5.94 


Write 


49.05 


8.10 


55.51 


14.26 


56.32 


9.31 


52.83 


10.72 


FSI Speak.* 


2.62 


.64 


3.13 


1.08 


1.97 


.66 


2.58 


.75 


FSI Read.* 


3.15 


.66 


3.10 


1.10 


1.89 


.57 


2.86 


.66 



Correlations with FSI Ratings 







"S" 


"R" 


"S" 


"R" 


"S" 


"R" 


"S" 


"R" 


MLA 


List. 


.67 


(.61) 


.73 


(.72) 


.84 


(.75) 




(.80) 


II 


Speak. 


.67 


(.49) 


.82 


(.83) 


.78 


(.66) 


.66 


(.65) 


II 


Read.' 


(.58) 


.71 


(.82) 


.82 


(.78) 


.69 


(.63) 


.74 


II 


Write 


(.65) 


^63 


(.86) 


.84 


(.62) 


.71 


(.70) 


.77 


FSI 


"R" 


(.69) 




(.95) 




(.90) 




(.80) 





* In computing these values, a "+" Is given a value of .5. Thus, 1+ Is coded 1.5, 
2+ - 2.5, etc. For the meanings of the FSI ratings, see below. 



Native or S-5 

bilingual 

proficiency 

Full S-4 

professional 

proficiency 

Hlmlmum S-3 

professional 

proficiency 



Limited S-2 

working 

proficiency 



Elementary S-1 
proficiency 



Speaking proficiency equivalent to 
of an educated native speaker. 



that R-5 Reading proficiency e- 
qul valent to that of an 
educated native speaker. 



Able to use the language fluently and 
accurately on all levels normally 
pertinent to professional needs. 



R-4 Able to read all styles 
and forms of the language 
pertinent to profession- 
al needs. 



Able to speak the language with suffi- 
cient structural accuracy and vocabulary 
to satisfy representation requirements 
and handle professional discussions 
within a special field. 



R-3 Able to read non-technl- 
cal news Items or tech- 
nical writing In a 
special field. 



Able to satisfy routine social 
demands and limited office 
requirements . 



R-2 Able to read Inter- 
mediate lesson material 
or simple colloquial 
texts. 



Able to satisfy routine travel needs R-1 Able to read elementary 
and mlmlmum courtesy requirements. lesson material or 

common public signs. 



"All the ratings except the S-5 and R-5 may be modified by a 
plus (■(') , Indicating that proficiency substantially exceeds the 
mlmlmum requirements for the level Involved but falls short of 
those for the next higher level." 



—Extracted from "Absolute Language Proficiency Ratings," 
Circular, May 1963, Foreign Service Institute, 
Washington, D.C. 



* The 6 Queens College senior cases were not used In this analysis 



T 



mm 
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It should be noted that this equation does not specify a regression line; that Is, it 
cannot be used to make optimal predictions of X from Y or Y from X. It n«rely assumes 
that X and Y are equally estimates of the same thing and that it is an arbitrary Mtter 
whether one measures this thing by X or by Y, Conceptually, this method of equating is 
similar to that employed by Melton and Myers (196A) . The more X and Y are correlated, 
the more this procedure is justified. It is felt that in the present case, the corite- 
sponding measurements are sufficiently well correlated to justify the procedure, parti- ^ 
cularly in view of the fact that the purpose of the study was merely to establish meaning<^ 
ful standards for the interpretation of the MLA scores. 

Table 2.3 shows the MLA equivalents (ln”conve'i#ed sco^’^ tot given FSI rating 
levels for the four tests in French, Germ^, Russian, ani Spanish; it also shows the 
linear equations Y ■ f(X) and X ■ f(Y). 

It will be noted that in a number of cases, the corresponding MLA scores exceed 
the maximum possible scores. This finding would suggest that the MLA tests in those 
cases do not have a high enough "ceiling,” that is, that they do not haw the capacity 
to discriminate among the upper levels of FSI ratings or indeed amo^ the upper levels 
of language competence (near-native and native Iraguage ability) • This matter e 

examined again when the distributions of MLA test scores from the national study are 

Inspected. 

Table 2.3 also shows the ranges of Superior, Good, Minimal, and Unsatisfactory 
rating equivalences established by Ityers and Melton (196A). It will be seen that in 
general in Listening, Minimal corresponds tc\ SI and Sl+ ji^ood to S2 to S3, and Superior 
to S3+ and above; in Speaking, Minimal extends from*Sltei. S^, Good from S2+ to S3+, and 
Superik>r to S4 and above; and in Reading and\also in Writing, Unsatisfactory corresponds 
to SI, Minimal to S1+ to S2, Good from S2+ to\s3+, and Superior to SA and above. 

There are some variations across languages, hiwdver: for example, in Russian, either 
the FSI ratings are unduly severe or the MyersVMelton ratings are unduly generous. It 
is largely a matter of opinion whether the FSl\equivalences established here aw more 
reliable than the Myers-Melton ratings. In fa^or of the FSI equivalences is the fact 
that they were based on a highly standardized plcocedure involving normal two-way 
communication between examiner and examinee; in\favor of the Myers-Melton equivalences 
is the fact that they were established on the badis of many more cases than were obtained 
for the FSI equivalences. Strictly speaking, the tfyers-Melton ratings apply only to 
scores on Form B of the MLA tests, but since con\Wrted scores were established to pro- 
vide equivalences between Forms A and B, the Myerb-Melton ratings should also apply to 

the Form A scores. \ 



3. The Design of the\Sample 



There were several reasons for desiring a large .and representative sample of 
college senior foreign language majors graduating at the end of the 196A-65 academic 
year. A large sample was desired not only because such a sample would yield resul s 
that the layman in statistics would be more likely to regSrd as convincing, but a so 
because if it were desired to break the sample down into various subgroups for special 
analvses, such subgroups would be larger than might otherwise be the case and hence 
would be more likely to produce statistically significant findings. The sample had to 
be representative in order to serve as an appropriate basis for the development of no™ 
that could be relied upon in the interpretation of scores in future administrations o£ 

the tests. 



In the original plan for the project, writteii early in 196A, the assumption had . 
been made, based on projections from enrollment data then availably that the total 
number of students graduating at the end of the academic year 196A-65 would exceed 
10,000 and that a representative sample of 7000 distributed proportionately amrag the 
five languages would be large enough to be persuasive. After the New York pilot study 
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Table 2.3 



MLA Poof iciency Test Equivalents (Y) and .Parameters of Linear Equations 

Relating Them to FSI Ratings (X ) \ 
with Ranges of Myers and Melton Ratings^ 



FRENCH 



FSI 


MLA 






FSI 


MLA 




Levels^ 


List. 


Speak. 




Levels 


Read 


Write 


SI 


32.2 M 


58.3 




R1 


2A.9 U. 


22.6 U a 


S1+ 


36.9 


66.0 M 




R1+ 


30.8 „ 
36.7 " 


28«7 


S2 


A1.6 


73.6 




R2 


3A.9 M 


S2+ 


A6.3 G 


81.2 




Ra-s- 


A2.6 


Al.O 


S3 


61.0 


88.8 G 




RT 


A8.6 G “ 


A7.1 


S3+ 


55.7 


96.5 




R3+ 


5A.5 


53.3 G 


SA 


60.A*S 


lOA.l 




RA 


60. A 


59. A 


S4+ 


65.2* 


111.7 S 




RA+ 


66.3 S 


65.6 « 


S5 


69.8* 


119. A 




1 


72.3* 


71.7 ^ 


Y-a+bX 


a 22.53 


A2.69 




a 


13.09 


10.39 




b 9.A8 


15.38 




b 


11.85 


12.27 


X-a*+b*Y 


a* -2.37A 


-2.773 




a* 


-1.105 


-0.8A8 




b» .105 


.065 


GERMAN 


b* 


.08A 


.082 


FSI 


MLA 






FSI 


MLA 




Levels 


List. 


Speak. 




Levels 


Read 


Write 


SI 


33.7 " 


59.0 

68.1 " 




R1 


30.5 U 


28.2 U 


S1+ 




R1+ 




34.7 


S2 


37. A 


77.2 
86. A 




R2 


40.5 ” 


41.2 “ 


S2+ 


A1.0 G 




R2+ 


A5.5 


A7.7 


S3 


AA.7 


95.5 - 




R3 


50,5 G 


5A.1 G 1 


S3+ 


A8.3 


10A.7 ^ 




R3+ 


55.6 


60.6 


SA 


52.0 . 


113.8 




RA 


60.6 


67.1 . 


SA+ 


55.7 ^ 


122.9 S 




RA+ 


65.6 S 


73.6 S 1 


S5 


59.3* 


132.1* 




R5 


70.6* 


80.1* 


Y«»fbX 


a 22. 6A 


AO. A3 




a 


20.51 


15.32 




b 7.3A 


18.36 




b 


10.03 


12.96 , 


X-a*+b*Y 


a* -3.083 


-2.206 




a* 


-2.0A3 


-1.180 




Table 2.3 continued 



RUSSIAN 



FSI 


MLA 




FSI 


MLA 




Levels 


List. 


Speak. 


Levels 


Read 


Write 


SI 


33.7 M 


68.9 ^ 
77,1 “ 


R1 




41.8 M 


S1+ 


37.9 


R1+ 


27.0 ^ 


49.9 


S2 


42.0 G 


85.4 G 


R2 


34.8 G 


58.0 G 


S2+ 


46.2 93.7 


R2+ 


42.6 


66.1 


S3 


50.4 


102.0 


R3 


50.4 


74.2 


S3+ 


54.5 


110,2 - 


R3+ 


58.2 - 


82.3* 


S4 


58.6*S 


118.5 ^ 


B4 


66.0 ^ 


90.4*S 


S4+ 


62.8* 


126.8* 


R4+ 


73.8* 


98.S* 


S5 


67.0* 


135.0* 


RS 


81.5* 


106.6* 



a + bX a 


25.48 


52.41 


a 


3.45 


25.45 


b 


8.30 


16.55 


b 


15.72 


16.33 


a*+b*Y a* 


-3.067 


-3.164 


a* 


-0.219 


-1.557 


b* 


.120 


.060 


b* 


.064 


.061 



SPANISH 



Y 



X 



FSI 


MLA 




FSI 


MLA 




Levels 


List. 


Speak. 


Levels 


Read 




Write 


SI 


36.4 “ 


71.2 " 


R1 


26.8 


U 


22.3 

30,5 


S1+ 


R1+ 


31.4 


M ■ 


S2 


40.1 


77.5 


R2 


35.9 


38.7 


S2+ 


43.9 G 


S?.H G 


R2+ 


40.5 




46.8.H 


S3 


47.7 


90.1 


R3 


45.0 


G 


55.0' f. 


S3+ 


51.5 


96.4 


R3+ 


49.5 




63.2 ^ 


S4 


55.3 S 


102.6 S 


R4 


54.1 




71.4 


S4+ 


59-. 0* 


108.9 


R4+ 


58.6 


S 


79.6 S 


S5 


62.8* 


115.2 


R5 


63.1 




87.8* 



a + bX a 


24.93 


52.22 


a 


18.03 


6.38 


b 


7.61 


12.65 


b 


9.00 


16.24 


a’+b*Y a* 


-3.272 


-4.125 


a* 


-2.003 


-0.394 


b* 


.131 


.079 


b* 


.111 


.062 



^Exceeds maximum possible score 

^For computational purposes, a ”+” Is given a value of .5; thus, 1+ Is coded 1.5, 
2+ ■ 2^5, etc. 

- Unsatisfactory; M - Minimal; G - Good; S - Superior 
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test Ine w»8 done, a proposal for the national study made In November, 1964 stated that 
In view of budgetary considerations a 50% sample, l.e. 5000 students, would be more 
practical. ThI final design of the sample was In any case not attempted until November, 
1964. when figures on 1962-63 college enrollments became available from the U. S. 

Office of Education making projections to 1964-65 more viable. At that time It was 
estimated that the 1964-65 numbers of graduating foreign language majors would be 
annroximately 12,700. This estimate was made on the assumption of a continuing and 
more or less constant growth rate evident In the following figures; 



Year 


FL Earned Degrees 


Annual Growth 


1960-61 


5451 (actual) 




— 


1961-62 


6714 (actual) 




1.232 


1962-63 


8334 (actual) 




1.241 


1963-64 


10,292 (projected from 
rate of 1.235) 


a growth 




1964-65 


12,711 (projected from 
rate of 1.235) 


a growth 





In the belief that this projection was perhaps too optimistic, a slightly more cot- 
aervaMve flcure of 12,000 was taken as the measure of the target population of the study, 
rthls estimate. It now turns out, was remarkably accurate. Official figures from the 
Ts! of Legion show 12.220 as the total nu>»her of 1964-65 graduates In the 

five languages.) Because previous decisions had Indicated the choice of 50% 
overall sampling fraction. It was decided to design a representative sample containing 
6000 siudiiL to be tested. The budgeting tapllcatlons of this decision were approved 
by the sponsors of the project. 

The samnllne design was developed. In consultation with Dr. Theodore Colton, a 
speclSlsHS s^IpJtnXory who hL been associated with 

i-ho Harvard University Department of Statistics and who was on the staff of the Depart 
»ent cr?revSuvrMedlclne In the Harvard Medical School. Basically, the design called 
fors?«Sfylng Se toial population by "size" of Institution l.e., by the total nu^er 
of students^majoring In the five test languages, as shown In figures for 

cost of teMlng at an Institution of a given size, (b) the variant 
scores within a stratum, and (c) the total budget 'rt^t^^e^of iSliull® 

(oSurvl“pXa^“^“hesrwi wL*onns“tutJ™s'belng prop« 

v!r^anLr;f ^»rwl\hinizrst«ta were estimated on the basis of data obtained In 
the New Tork pilot study (and found to be approximately equal) . Helevant dlscMslon 
f^Sas pertLnlng^o the sampling design are to be found In Cochran (1963. p.95f, 

289) and Hansen, Hurwltz, and Madow (1953, pp. 220f.) 

"Public?' institutions are defined as those supported and administered under the 

oS endLnents or private funds, e.g. Princeton Hnlverslty. Goucher College. 

A..h,iallv the sample was designed In several stages because of a nusiber of co^ll- 
catloS?^af ln™LutloSal sizes were In terms of 1962-63 data ™d would 

Sve to’bi ggradlusted aU» Ir 

:iSeTo’^Tore”:r^^^^^^^ 1964-63 fig-- ^ Che i^tltu- 

sSp^s^ran^d^stui^d s:rtMu.d 

for by the drawing of a supplementary sample. 
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Five "size" categories of institutions were established somewhat arbitrarily in 
!fuch a way that there would be roughly equal numbers of students in the total popula- 
tion in each. The basic data available for designing the sample are shown in Table 2,4, 
the number of public and private institutions in the five categories, and the number of 
students who earned degrees in each. The sampling design did not involve taking account 
of the numbers of students in the separate languages, it being assumed that random 
factors would result in an approximately proportional distribution, over the five 
languages, in any properly chosen sample. 



Table 2.4 



1962-63 Population of U. S. Institutions 
Offering A. B. Degree Programs in 
Test Languages, with Ntraibers of Earned Degrees 
Source: U. S. Office of Education 



Size category 


Stratum 


Public 


Institutions 


Private 


Institutions 


Totals 


(No. of students 
in all languages) 




No. of 
Institu- 
tions 


No. of 
Earned 
Degrees 


No* of No. of 

Instltu- Earned 
tions . . Degrees 


No. of 
Insti- 
tutions 


No. of 
Earned 
Degrees 


40 and over 


5 


25 


1881 


11 


674 


36 


2555 


20-39 


4 


27 


746 


43 


1122 


70 


1868 


10-19 


3 


45 


614 


93 


1240 


138 


1854 


5-9 


2 


61 


418 


134 


895 


195 


1313 


1-4 


1 


75 


179 


257 


575 


332 


754 


Totals 




233 


3838 


538 


4506 


771 


8344 



First sampling allocation 

The goal of the sampling design, in every case, was to determine sampling rates 
for each size stratum such as to minimize the error in estimating a population mean 
of a test variable, under a given budgetary restriction and under given assimptions as 
to the cost of testing. A first determination of sampling fraction® was made on the 
assumption that if the study had been done in 1963, a 60% sample might have been dra^ 
containing 5000 students, and that such a study would cost $17 per student plus $25,000 
for institutional and administrative costs, or a total of approximately $110,000 a 
figure that was in the neighborhood of the amount budgeted for testing. It was thought 
that with optimal sampling allocation, the institution costs might be reduced. It was 
planned also, after this sampling allocation, to re-scale the sample to allow for pro- 
jected 1964-65 figures. 

First, a cost function was established as 

5 

$ 110,000 - + $17 \) % 



- 20 - 



where a - administrative cost for obtaining each institution^jLn the ^ 

M. - av«age number of senior FL majors per i^titution in the h s ^ 

SSmber of fnstitutlons in the sat^ple of the h“ stratum. *»ere being strata. 

It was assumed that the second-stage sampling rate would be 100%, i.e. th^ 

n, majors within a given institution would participate in the testing. Thus, the p ■ 

Ing design may be regarded as calling for a clustered sample. 

The solution for values of tL such that the variance of a mean test score is mini- 
mized subjeft to the total budget"of $110,000 is given by 




where 



ttl 

y . ■ proportion of students in the h stratum, 

th 

S 2 m Variance between means of schools in the h stratum, and 



ttl 

y as total number of institutions in the h stratume 



If we assume that is constant for all h, we get 



"h 



W, 






Z (W„/ -b 917^ > 

$110,000 



^ 4 1 -Witt •!« Stratume Therefore* the sampling fraction for this 

institutions available in this stratum. . ,7 „ 9555) - $45,667, 

stratum is taken to be 100*. This stratum would cost $(62 x 36 + 17 x ^333j , 
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leavlng $64 » 333 as the total budget remaining. The remainder of the worksheet is a 
reallocation of the other four strata, giving the final sampling fractions shown in the 

last column. 



lit was noted, however, that while the optimal allocation procedure would yield a 
sample of 5825 students, it required testing at 312 institutions and did not take into 
account the growth of student bodies that would undoubtedly have occurred in those in- 
stitutions. Basing the sample oh these computations would risk getting too many stud- 
ents— more than our budget would allow for. Consequently, a further set of comput^lons 
was performed (Table. 2.6) on the assumption that all student bodies (l.e., senior FL , 
majors) would in 1964^65 be 156% of the figures for 1962-63. This was arrived at on the - 
assumption of an annual growth ratio of 1.25 applied for two years. Costs par school 
were reflgured to allow for population growth. The resulting allocation required a total 
of 188 Institutions, with sampling rates as shown in column 12 of Table 2.6. Computa- 
tions showed that the estimated number of students to be tested according to this 
sampling planvTrould be 6014, at an estimated cost of $109,993, a figure close to the 
budgeted figure. Because this sampling plan was based on a liberal estimate of popu- 
lation growth. It was actually conservative with respect to the budget because It pro- 
tect* ^ us from having to test an unexpected number of students at too large a number of 

institutions. 



The next step was to apply the sampling rates to the numbers of institutions in 
Table 2.4 in such a way that "public" and "private" institutions would be proportionately 
represented in each stratum. Because the sampling rates were adjusted to actual frac- 
tions (4/5, 1/2, etc. In col. (13) of Table 2.6) to facilitate the use of random number 
tables in the drawing of the sample of institutions from lists, the final , •? 

Institutions in the various strata, given in the "number invited column of Table 2.7, 
are not always exactly equal to the numbers specified in the allocation • 

It was on the basis of Table 2.7 that a mailing was made on December 10, 1964 to 192 
institutions inviting them to participate in the study. In 1962-63, the total number o 
FL majors in these 192 institutions was 3,869. Using the previously explained growth 
adjustment of 156%, it was estimated that there would be approximately (^000 students in 
the sample An 1964-65 if all institutions and all students responded. This number 
suited both the plans and the budget. 



Second sampling allocation 

On January 17, 1965, the principal investigator felt that it was necessary to re- 
consider the sampling design in view of indications ^hat, the institutional 
would not be as high as had been hoped for. Afii of January 14, r p 

from 147 (76.6%) of the 192 institutions originally invited to participate (see Table 
2.7). Of the 147 instltutjlons heard from, 124 (84.4%) responded favorably. 
these stated that some but not all of its foreign language 

pate.) The 124 institutions responding favorably constituted 64.6% of the 192 instl 
tutlons invited. The total actual number of students reported by the favorably- 
responding institutions was 3678, or 61.3% of the 6000 students projected. for the sample. 
By comparing this number with the projected number (4040) we would 

tLse institutions on the basis of the projection ratio of 1.56, a new estimate of the 
actual two-year growth was possible, namely 1.42. 

A redesign of the total sample, using the new growth ratio 
possible. It was assumed that all institutions invited would eventually be heart fr^, 
prompted if necessary by reminder letters and telephone calls. It was 
that the rate of favorable reply among institutions varied according to strata, but not, 

apparently, according to the public-private variable. Small gL-t,™ 

l iLiy to reply favorably than large institutions. Account was taken of this variatrtn 
by appropriate^adjuatmenta in sampling rates after the reallocation computations shown 

in Table 2.8. 

Further, all sampling rates were multiplied by 1.30 In order to allow for the non 
reply rate of 77% established on the basis of the number of replies received as of 
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Worksheet for First Sampllfig Allocation Adjusted for Projected Growth 
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Table 2.7 



Initial Sampling Design, and 
Institutional Response as of January 14 , 1965 



Size 



40 + 

20-39 

10-19 

5-9 

0-4 

Total 



40 + 

20-39 

10-19 

5-9 

0-4 

Total 



40 + 

20-39 

10-19 

5-9 

0-4 

Total 



No. 

Invited 



No. 

Replying 



PUBLIC INSTITUTIONS 

20 14 

14 9 

15 15 

13 12 

9 2 ^ 

71 59 

PRIVATE INSTITUTIONS 



9 

22 

31 

30 

29 

121 



29 

36 

46 

43 

38 

192 



6 

18 

22 

26 

16 

88 

TOTAL 

20 

27 

37 

38 
25 

147 



% 


No. 


% 


plying 


Replying 


Favorable 




Favorably 


Replies 


70 


14 


100 


64 


8 


89 


100 


14 


93 


92 


9 


75 


100 


6 


67 


83 


51 


86 


67 


5 


83 


82 


18 V 


100 


71 


19 


86 


87 


19 


73 


55 


12 


75 


73 


73 


83 


69 


19 


95 


75 


26 


96 


80 


33 


96 


88 


28 


74 


66 


18 


72 


77 


124 


84 
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January 14. This figure waa applied uniformly to strata since there seemed to be no 
consistent correlation between stratum and reply rate (see Table 2.7). (Through computa- 
tional error, however. It was not applied to stratum 2; see Table 2.9 for further 

details.) 

These final sampling rates were applied to the data of Table 2,4 In order to specify 
the numbers of Institutions needed In a final, "expanded” sample as shown in Table 2.9 
and the nuod>ers of Institutions that had to be added to the previous sample to achieve 



Table 2.9 

Final Sampling Design 



(Numbers of Institutions) 







PUBLIC 






PRIVATE 




Stratum 


Pop. 


Orlg. 

Sanple 


Added 


Expanded 

Sample 


Pop. 


Orlg. 

Sample 


Added 


Expanded 

Sample 


5 


40+ 25 


20 


5 


25 


11 


.‘9 


2 


11 


4 


20-39 27 


14 


5 


19 


43 


22 


8 


30 


3 


10-19 45 


15 


9 


24 


93 


31 


15 


46 


2 


5-9 61 


13 


7 


20* 


134 


30 


13 


43* 


1 


0-4 75 


9 


8 


17 


257 


29 


29 


58 




233 


71 


34 


105 


538 


121 


67 


188 










TOTAL 










Stratum Size 


Pop. 


Orlg. 

Sample 


Added 


Expanded 

Sample 








5 


40+ 


36 


29 


7 


36 








4 


20-39 


70 


36 


13 


49 








3 


10-19 


138 


46 


24 


70 








2 


5-9 


195 


43 


20 


63* 








1 


0-4 


332 


38 


37 


75 












771 


192^ 


101 


293 






*These are the actual figures used. It was later discovered, however, that ^ 
putatlonal error had been made. The correct figures for the expanded sample should have 
been: Public, 25; Private, 56; Total, 81. 


the expanded sample. In all, 101 Institutions had to be added, according to these com- 
putations, and random sampling procedures were applied to select from a list of Institu- 
tions for a "second wave" of Invitations. Actually, mailings were made to 101 Ittstltu- 
tlons on January 21, 1965, making a total of 293 Institutions In the final sample. As 
of 1962-63, there was a total of 5353 students who earned degrees In the five foreign 
languages; applying the growth ratio of 1.42 to this figure, we get * total of 7601 
students In the estimated target population. It was not expected, however, tnat aii 



*Although It did not, /happen to appear In the random sample drawn. Harvard and Rad 
cllffe Colleges were also participating Institutions. As had been true the previous 
year, they contributed very few students to the sample due to student non-response. 



o 
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these students would be tested because of both Institutional and student non-response 
rates. Chapter III presents and discusses the outcoiiies of the sampling design. 



4. Procedures In Data Collection 



The success of this study depended in large part on the excellent cooperation of 
many Individuals at numerous institutions of higher learning in the United States. They 
were mainly responsible for the actual collection of the data. A great deal of admin- 
istrative work, however, had to be done at the project headquarters in order to insure a 
smoothly running program. Too, a considerable amount of work was done by personnel at 
Educational Testing Service in shipping test materials and scoring the test results. 

This section describes the sequence of events planned and carried out by all parties 
concerned . 

(1) Once an institution was selected for participation in the study, the project 
li'eadquarters at Harvard mailed to the president or chief executive of that institution 
the following items: (these items, except for reply envelopes and the like, are in- 

cluded in this report in Appendix D) 

a. Five copies of a covering Memorandum to Presidents of Inst itutions of Higher 

* Learning signed by Dr. Kenneth Mildenberger , then Director of the Division of 
College and University Assistance, U. S. Office of Education, pointing out the 
need for and the importance of the project and encouraging the institution to 
participate; it requested each President to appoint a coordinator for the test- 
ing project at his institution; 

b. Five copies of a Memorandum Describing the Project, that could be distributed to 
foreign language department chairmen and others who might be concerned with the 
decision whether or not the institution would participate in the project; 

c. An Institution Information Form asking whether the institution would participate 
in the project, in what languages the institution offered "bachelor’s degree 
programs" and how many seniors were currently "majoring" in each language; if 
the institution had decided to participate in the testing program, the form also 
asked for the name and mailing address of the person who had been designated as 
coordinator for the program and for the names and addresses of each FL depart- 
ment chairman. An early deadline was given for the return of the form. 

d. A business reply envelope for return of the Institution Information Form. 

Comment: The decision to mail materials directly to the chief executives of the 

institutions concerned was based partly on the fact that no comprehensive list of for- 
eign language department chairmen was available. Furthermore, it was reasoned 
presidents should have information about the project and have th' right to make the 
final decision as to the participation of their institutions. In the New York State 
pilot study, much more use of direct communication with chairmen of foreign language 
departments was employed, with perhaps slightly better results. Ctae faces a fundamental 
dilemma in a study of this type: while the chief executives of iristitutions are the 

proper authorities to address, for many reasons, they are busy people and frequently 
have poor communication with their own department chairmen, who in many cases would have 
more interest in such a study than the presidents. 

As noted earlier, the first wave of mailings was made to 192 institutions on Decem- 
ber 10 1964, and the second wave was made to 101 institutions on January 21, 1965, in 
anticipation of testing dates of institutions that would occur as early as possible in 
the spring of 1965. It is impossible to say whether higher institutional reaponse rates 
would have occurred if it had been possible to make the mailings earlier. 
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(2) On receipt of the above kit of materials and after deliberation as to partici- 
pation in the study, the institution returned the completed Institution Information Form 
to project headquarters at Harvard. 

Comment: The fact that this form was to be sent directly to Harvard rather than to 

ETS reflected the decision to centralize as much as possible of the administrative work 
at Harvard in order to make coordination more effective and to minimize administrative 

charges . 



(3) Beginning January 15, 1965, upon receipt of the Institution Information Form 
from institutions that had decided to participate, the project headquarters mailed to the 
designated institutional Coordinator the following kit of materials: 



a. (Mimeographed) "Instructions and Suggestions to Coordinator for Administering 
Test Materials under the College Foreign Language Testing Project, Spring 1965 
—a "master information source for all activities and responsibilities of the 
institution ^coordinator" (Appendix D-4) ; 

b. One copy of "Order Form for Administration of MLA Foreign Language Proficiency 
Tests for Teachers and Advanced Students under the College Foreign Language 
Testing Project "—to be returned to project headquarters at Harvard in an air- 
mail business reply envelope that was supplied (Appendix D-5) ; 



c One "Application for Reimbursement of Coordinators and Assisting Proctors"— to 
’ be returned to project headquarters after the administration of the tests; 



d. 



A sufficient supply of the booklet "A Description of the MLA Foreign Language 
Proficiency Tests for Teachers and Advanced Students (ETS, 1964a) to distribute 
to each foreign language major graduating from the institution in the spring of 
1965 in one of the five test languages; 



e. An equal number of copies of the leaflet "A Letter to College Seniors Graduating 
in the Spring of 1965 with Degrees in French, Italian, Spanish, German, or 
Russian" — also to be distributed to students (Appendix D-6) ; 



f. An envelope containing materials to be distributed by the Coordinator to foreign 
language department chairmen; it contained for each department chairman (1) a 
copy of the above-mentioned "Memorandum Describing the Project (Appendix - ), 
(2) the "Questionnaire for Chairmen of Foreign Language Departments (Appendix 
C) , and (3) a business reply envelope for the use of the chairmen in returning 
the questionnaire to project headquarters. 



Comment: The Coordinator had the responsibility of arranging the testing dates, 

distributing materials to department chairmen, informing students about the 
seeing that they were informed, ordering fast supplies, arranging for the actual admin- 
istration of the tests, and returning materials either to ETS (the completed tests 
materials) or to Harvard (the student questionnaires and the vouchers for payment of 
himself and proctors). For this he was paid $30; at the larger institutions this was 
probably hardly adequate to recompense the coordinators for the amount of time they a 
to devote to the project. Coordinators at the larger institutions were permitted o 
employ assisting proctors, tc be paid $10 apiece. 

It was specified that testing dates were to fall within the period March 1 to May 
8, 1965 and were to be set in such a way as to allow sufficient time for testing mater- 
ials to be shipped to the institution from ETS. 



The Coordin'>tor was conceived to be the key factor in encouraging students to par- 
ticipate in the testing. It was recommended to Coordinators that th«y 
studLts personally. There is no information available as to the extent to which they 
actually did so.' As will be seen, student response varied widely from institution to 

institution. 
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(4) Upon receipt of the test order form, the project headquarters arranged with ETS 
to send to the Coordinator a sufficient supply of MLA Proficiency Tests (in the relevant 
languages), Modern Language Aptitude Tests, and Student Questionnaires to cover the 
numbers of students expected to be tested. (A small excess supply was provided in ac- 
cordance with ETS experience in this matter.) The packages of test materials xncluded 
directions for administering the tests. Sent directly from Harvard to each institution 
was a copy of Directions for Administering the Questionnaire for Foreign Language Majors 

(Appendix D-7) . 

(5) The actual distribution of weeks in which testing dates selected by institutions 
fell was as follows : 

March 1-6 1 

March 8-13 5 

March 15-20 ..... 13 

March 22-27 19 

March 29-April 3 . . 31 

April 5-10 50 

April 12-17 17 

April 19-24 34 

April 26-May 2 ... 27 

May 3-8 12 

May 10-15 ..... . . 2_ 

211 

The excess total number in the above table (as compared to the 202 institutions in the 
study) reflects the fact that a few institutions scheduled different parts of the test- 
ing program in two diffarent waeks. 

Normally, all students who volunteered to participate in the study took the four 
"skills" tests of the MLA Proficiency Series in the language they claimed as their major 
language, the "short form" of the Modem Language Aptitude Test, and the Student Ques- 
tionnaire. There were no specifications as to the order in which these tests were ad- 
ministered except that the Listening and Speaking tests of the MLA series were to be 
given always in that order. Students enrolled in teacher preparation programs also took 
the three professional knowledge tests of the MLA series . 

(6) Upon termination of the testing, the MLA and MLAT materials both used and 
unused— were returned to ETS, where the answer sheets were scored, and the Student Ques- 
tionnaires were returned directly to Harvard, where the information contained therein 

was key-punched. 

(7) ETS supplied Harvard with cards containing the scores for the tests. Because 
of the considerable amount of time required to score the Speaking tests (which requxred 
students to record their responses on tape) , ETS was not able to supply the bulk of the 
scores until September, 1965. However, special arrangements had been made to score the 
tests of students at institutions in Pennsylvania as early as June so that the scores 
could be supplied to department chairmen at those institutions for use in the state of 
Pennsylvania teacher certification program. The scores supplied by ETS were c^yerted 
scores" on the MLA Proficiency Tests, but- raw scores on the three parts of the MLAT that 

were administered. 

(8) Using the addresses that the students had supplied on the Student Questionnaire 
(Appendix B) , the project personnel made up individual score reports and mailed them to 
the students on October 7, 1965. The score reports were in the form of pressure- 
sensitive labels, imprinted by tabulating equipment, that were affixed to a Memorandum 
to the Foreign Language Majors who Took the MLA Foreign Language Proficiency Tests for 
Teachers and Advanced Students in the Spring of 1965 (see sample. Appendix E-1). Each 
score report was accompanied by an interpretative leaflet to enable the student to 
assess his performance in comparison to norms compiled for NDEA Instxtute students ( p- 
pendix E-2) . Scores on the Modern Language Aptitude Test were not furnxshed to students; 
it was explained that these scores were collected only for statistical control purposes. 
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(9) Rosters of students' MLA test scores were made up at the project headquarters 
and mailed to foreign language department chairmen on October 22, 1965 with a covering 
letter (Appendix E-3) and interpretative leaflet (Appendix E-2) . 

(10) Department Chairmen's Questionnaires were returned directly to Harvard by the 
department chairmen themselves. 

(11) Data collected were further processed for analysis at project headquarters. 
Various statistical tabulations were made over a period of at least a year because of 
the large volume of data and the many cross-comparisons and correlations that were 

desired. 

(12) The data have been archived on magnetic tape and are deposited with the ERIC 
Clearinghouse on Foreign Languages, Modem Language Association, 62 5th Ave., New York 
IDUll, N.Y. foruseby qualified investigators. The tapes are accompanied by appropriate 
documentation as to card codes and formats. 

Throughout the above steps, it was frequently necessary to send reminder letters 
to the various key persons involved in order to secure the best possible response. 



o 
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Chapter III 



OUTCOMES i OF THE SAMPLING DESIGN 



This chapter is concerned solely with general matters pertalnlnR to the amount 
and character of data accumulated as a result of the sampling design and the procedures 
for gathering data. It does not attempt to estimate the amount of bias In the sample or 
correct for It; this Is done In Chapter V. 



1. Institutional response rates 



According to Dr. Peter Loret of Educational Testing Service, who participated In a 
June 1966 conference to Inspect the preliminary results of this study, ETS had^^found that 
the Institutional response rate could be regarded as a more or less permanent "institu- 
tional characteristic." Some Institutions are consistently willing to participate In 
educational testing surveys, while others are consistently unwilling. In his experience, 
an overall response rate of 50% was typical. 

In the present study, 203 of the. 293 institutions Invited to participate did so; the 
Institutional response rate was thus .69.3%. Detailed figures are presented, by strata 
and the public-private dichotomy. In Table 3.1. Although there was little difference 
between public and private Institutions in rate of response, there was a clear trend 
whereby the larger Institutions were more likely to agree to participate. 

Comparison with the response rates obtained In. the New York pilot study of 196A are 
of Interest (Carroll, 1965). The Invitations to Institutions were mailed out much later 
In the year In that study— on March 24, 1964, and the testing dates were typically In 
early May. Still, the overall institutional response rate was quite good — 70.1%. It had 
been concluded that it might have been better If the Instltutlcns had been Invited much 
earlier In the year, and It was on this basis that the present study began Inviting In- 
stitutions early. Nevertheless, the national Institutional response rate was no better 
than that of the New York pilot study. A possible explanation for this Is the keen In- 
terest of New York Institutions In the MLA Tests In connection with teacher certification 
requirements, as compared with the general level of Interest In the MLA tests throughout 
the country. (In the national study, the response rate of New York Institutions was77.4%j 

Response of Institutions to the mailings was fairly prompt. All the favorable re- 
plies were received within eleven weeks of the initial mailing. Replies to the first 
wave of invitations wer*) somewhat prompter than those to the second, and also, a higher 
proportion of favorable replies were received to the first :;?ave of Invitations (146 out 
of 192, or 76.0%) than to the second (60 out of lOl, or 59.4%). Table 3.2 gives the 
details. 

Five Institutions Initially agreed to participate, but later found It necessary to 
withdraw. In the tabulations presented here,. they are treated as non-participants . As 
of April 9, 1965, only 10 Institutions had not given a reply — even after reminder letters 
and telephone calls (1 from the first sample, 9 from the second sample). Further efforts 
to get these Institutions to participate were abandoned since It was by then too late to 
arrange for testing. 
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Table 3.1 



Institutional Response Rates 



Stratum 



Size’ 



No. in 
Population 



SOLICITED 
FINAL SAMPLE 



N 



% of 

Population 
(Actual 
Sampling 
Rates) 



N 



PARTICIPATING 



% of 
Sample 
(Response 
Rate) 



% of 

Population 

(Net 

Sampling 

Rate) 



PUBLIC 



5 


40+ 


25 


25 


100.0 


20 


80.0 


80.0 


4 


20-39 


27 


19 


70.4 


.13 


68.4 


48.1 


3 


10-19 


45 


24 


53.3 


17 


70.8 


37.8 


2 


5-9 


61 


20 


32.8 


11 


55.0 


18.0 


1 


0-4 


75 


17 


22.7 


11 


64.7 


14.7 


Total 




233 


105 


45.1 


72 


68.6 


30.9 










PRIVATE 








5 


40+ 


11 


11 


100.0 


10 


90.9 


90.9 


4 


20-39 


43 


30 


69.7 


26 


86.7 


60.5 


3 


10-19 


93 


46 


49.^ 


32 


69.6 


34.4 


2 


5-9 


134 


43 


32.1 


30 


70.0 


22.4 


1 


0-4 


257 


58 


22.6 


33 


56.9 


12.8 


Total 




538 


188 


34.9 


131 


69.7 


24.3 










TOTAL 








5 


40+ 


36 


36 


100.0 


30 


83.3 


83.3 


4 


20-39 


70 


49 


70,0 


39 


79.6 


55.7 


3 


10-19 


138 


70 


50.7 


49 


70.0 


35.5 


2 


5-9 


195 


63 


32.3 


41 


65.1 


21.0 


1 


0-4 


332 


75 


22.6 


44 


58.7 


13.3 


Total 




771 


293 


38.0 


203 


69.3 


26.3 



*Number of FL majors in all five languages as of 1962-63. 
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Table 3.2 

Rate of Return of Favorable Replies 
from Institutions 





First 


wave of Invitations 


Second 


wave of Invitations 




(192 Mailed Dec. 10, 


1964) 


(101 Mailed Jan. 21, 


1965) 


Wks. 

after 

mailing 


Wk. ending 


No. of 
favorable 
replies 


Cumulative 
per cent 


Wk. ending 


No. of 
favorable 
replies 


Cumulative 
per cent 


1 


Dec. 18 


7 


3.6 


Jan. 29 


0 


0.0 


2 


Dec . 25 


47 


28.1 


Feb. 5 


17 


16.8 


3 


Jan. 1 


26 


41.7 


Feb . 12 


19 


35.6 


4 


Jan. 8 


13 


48.4 


Feb. 19 


14 


49.5 


5 


Jan. 15 


23 


60.4 


Feb. 26 


4 


53.5 


6 


Jan. 22 


2 


61.5 


Mar. 5 


1 


54.4 


7 


Jan. 29 


0 


61.5 


Mar. 12 


1 


55.4 


8 


Feb. 5 


5 


64.1 


Mar. 19 


3 


58.4 


'9 


Feb . 12 


16 


72.4 


Mar. 26 


1 

\ 


59.4 


10 


Feb. 19 


5 


75.0 




\ 

\ 

\ 




11 


Feb . 26 


2 


76.0 










Total 


146 


76.0 




60 


59.4 
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2. Student response rates 

It will be recalled that the design of the sample called for a two-stage sampling — 
a stratified optimal-allocation sample of institutions at the first stage and a lOOZ 
sample of students at the second stage. Every effort* was made to encourage all gradua- 
ting FL majors at the selected participating institutions to take the tests, but despite 
these efforts, the student response rates were far short of what was desired. 

Evidence as to the numbers of students available for testing at the institutions 
was available from two sources. (1) figures provided by the departments themselves in 
connection with the administration of the tests (see Item 2, Institution Information Form, 
Appendix D-3), and (2) official figures on earned degrees at U. S., Colleges and universi- 
ties collected by the U. S. Office of Education and generally published one or two years 
after the graduation date (for example, see Tolliver, 1963a, 1963b; Wright, 1965, 1966). 
As of this writing, the official figures for the academic year 1964-65 have not been 
published^, but tabulator cards containing the relevant information were supplied to 
the project in July, 1966. In view of the fact that these latter figures are of an 
official character and cover both participating and non-participating institutions, it 
was decided to rely on these, rather than those supplied by department chairmen, to make 
an analysis of student response rates. (Reliance was placed in the accuracy of the data 
in the cards; it may eventually be found that there are slight discrepancies with pub- 
lished figures.) 

In making this analysis, however, it was necessary to exclude students at institu- 
tions that had not been included in the 1962-63 population of institutions from which 
the sanq>le had been drawn, since only in this way could the success of the sampling de- 
sign be fairly judged. The U. S. Office of Education figures showed a total of 12,220 
students as having earned B. A. degrees in the academic year 1964-65, but only 11,633 of 
these were in institutions defined as being in the 1962-63 population of institutions. 

The analysis of student response rates had to take account also of the fact that an 
examination of the total CFLT sample utilized in various tabulations later in this report 
included 3 students fested at Harvard University, an institution that had not been drawn 
in the random sample of institutions but where testing was offered as a matter of 
courtesy. (The 3 students represented, actually, only a small fraction of those availa- 
ble to be tested at the institution.) The final count of graduating senior students 
legitimately tested at institutions that had been drawn in the sampling design was 2784. 

From the U. S. Office of Education figures, it was determined that there were 5427 
students available to be tested in 1965 at the institutions participating in the study. 
The overall student response rate, therefore, was 51. 3Z. The tested students represent- 
ed, in all, 23. 9Z of the 11,633 students in the target population of French, German, 
Italian, Russian, and Spanish majors in the 1962-63 population of institutions. 

Table 3.3 shows a detailed breakdown of the student response rates by size-stratum, 
institution type, and sex. Several sets of percentages may be considered. First we 
may consider the percentages that the FL majors in the participating institutions con- 
stituted of the total target population as defined above. These percentages reflect the 
sampling design whereby larger sampling fractions were used for the drawing of the larger 
Institutions. They also reflect the institutional response rates discussed in the pre- 
vious section. If all students at the participating institutions had volunteered to take 
the tests, as the sampling design had contemplated, these percentages would have reflect- 
ed the degree to which each stratum of the target population was represented in the 
tested sample. They would have deviated from the optimal sampling design only to the 
extent that institiational response rates varied. It can be seen that the "available 
student" sub-population was somewhat more representative of students at public institu- 



^The citation to the publication, in press at this writing has been furnished by 
the U. S.i Office of Education. See Hason and Rice (1967), in the References. 
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6SV this difference Is escribable to differences between public 
J in the distribution of students over strata of different sizes. The s^le 

about equally representative of male and female foreign language majors, 
despite the preponderance of females in the target population. 

rid: Of is the next set of percentages, showing the actual student, response 

rates in”insti tut ions of different sizes and types. Three major conclusions emerge 
from these figures: 

(1) There was no appreciable or consistent difference in the response rates of 
■an aa compared to women. (See Technical Note below.) 

121 There was a strong negative relationship between response rate and Institution 
1 .^rti, wM tme in both public and private Institutions. That Is, student response 
rites iiii rilailiili 1« the larger institutions and relatively high in the smaller 

institutions . 

<„ r sT.;.r: r.i’ir..:: s 

to take the tests in the study. 

Technical note: The differences in reponse rates by sex, institution-type 

end sini^ ier e i laminable for statistical significance by the chi-square test. 
iS^the case of the sex variable, the value of chi-square for the total sample wm 
2 65 with 1 d.f., not significant at the 5Z level even though there wm a slight 
iiKweiie favwiig the w^en. (The chi-square test was also applied to sex 
differences in response rates within particular strata and institutiTO-typ 
classifications. Significant differences favoring the women at the 5* 1 
Se«er w«e fou^d in Public Stratum 4. Public Stratum 1, and Private Stratum 3. 
while a si" vlflcant difference favoring the men was found in Public Stratum . 

TOese Hnkngs rnot add up to any consistent pattern and »ay therefore be 
A A variations of no practical Interest •) The chi squar 

Jtl«f for the public-private dichotomy and the institution-size 

both very highly significant, 404.2 with 1 degree of freedom, and 806.0 

with A degrees of freedom, respectively. 

mu 1 a. ai-tffgzrpnces in Student response rates between institution types and sizes 
The large differences in student ^ validity of the 

aamle°^burthere\as nothing that could reasonably have been done post fa|t£ to alter 
sample, but there was » placed on statistical analyses of the data 

ttat“^e“at Ld^^order to estimate how much student response-rate differences might 

have biased the sample. This problem is taken up in Chapter V. 

Th. final set of percentages listed in Table 3.3, in the last three columns, shw 

"uL“’‘The‘’ll^i fl^ie in the table, for e:^le. is 24.0Z, indicating that 
24X*of 11^3 k majors in tL target population appeared in the CFLT 

U nft o? cou J. that the 24Z appearing 

representative of the intention of the samp- 

HSiSor mHI; was aSU^d'o^ ^^rial^yl^t'v-’i-stra^^^ 
response-rates that were lower than desired. 

Actually, the combination of ^"rirtUufioS^slfu- 
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The net effect is that the CFLT sample has the appearance of being sampled from stratum, 
institution-type, and sex classifications with approximately equal sampling fractions 
(ranging only from 15. 9Z to 27. 9Z for the 5 strata when institution-types are combined). 

If (and this is a ”big if") student response rate is uncorrelated with the dependent 
wiables of the study, the CFLT sample can be used as it stands with no correction for 
student response-rates or sampling fractions in the sampling design, as reasonably repre- 
sentative of the target population. As will be shown in Chapter V , it is unlikely that 
student response rate is highly associated with the chief dependent 

study (the MLA skills tests) , although there is probably a spal l degree of association. 

For this reason we feel that there is probably little error introduced by “fing the CFLT 
sample as it stands, for many of the analyses and interpretations made in this study (as 
is done in all chapters except ChapterV). 

Table 3.4 shows an analysis of student response by language. A chi-square test 
shows that there is a highly significant (p <.001) difference in these response rates. 

The difference, however, appears to be largely associated with Russian, where the re- ^ 
sponse rate was consistently low in both public and private institutions. No suggested 
explanation can be offered for this finding, nor can we suggest what implications it has 
as to possible special biases of the Russian sample. 

3 , Completeness of Data and Prejaration of Sample. 

In a study of this type, conducted with the cooperation of many different persons 
at institutions spread throughout the U. S., it is inevitable that the data are not as 
complete as might be desired, even for those students who actually participated in the 
study. The testing program was long, and it was probably difficult for institutional 
coordinators to insure that all participating students took every test and filled out a 
questionnaire. "Missing data" pose a problem for the statistical analyst. For some 
types of questions, he attempts to estimate what the data would show if none were missing. 
For other types of questions, it suffices to analyze the data at hand and assuro that 
they are representative of what might have been obtained if the data were complete. 

There were three basic instruments for students in the present study: the MLA Pro- 
ficiency Test in the relevant language, the Modem Language Aptitude Test, and the Stu- 
dent Questionnaire. When all the data were all assembled, it was found that there were 
2880 different individuals who contributed data on some or all of these instruments. 





N 


% 


French 


1300 


45.1 


German 


438 


15.2 


Italian 


31 


1.1 


Russian 


119 


4.1 


Spanish 


992 


34.4 




2880 


100.0 



Examination of these data, however, showed that 58 (2.0%) were underclassmen and 
35 2%) had already graduated and therefore should not have been tested. These cases 

were eliminated, leaving 2787 legitimate cases. All these cases had student question- 
naire data, except that for a few of these, there were missing entries in the question- 
naires . 

Of the 2787 legitimate cases: 

i 

2587 (92.8%) had scores on all four skills tests (Parts 1-4) 

193 (6.9%) had some scores missing on Parts 1-4 (most of the missing scores 
being for the Speaking Test because of poorly recorded tapes) 

7 had no scores on MLA skills tests (of these, 2 had scores on the teacher 
preparation tests) 
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Table 3.4 

Student Response Rates, by Language 



No. in No. in Institutions Tested Students 

- a/ o/ 



Language 


Population 

(*64-*65) 


of Final Sample 
N % 


N 


% of 
Sample 


% of 

Population 








PUBLIC 








French 


2231 


1171 


52.5 


496 


42.4 


22.2 


German 


930 


535 


57.5 


188 


35.1 


20.2 


Italian 


53 


40 


75.5 


10 


25.0 


18.9 


Russian 


290 


206 


71.0 


55 


26.7 


19.0 


Spcuilsh 


2314 


1209 


52.2 


504 


41.7 


21.8 


TOllAL 


5818 


3161 


54.3 


1253 


39.6 


21.5 








PRIVATE 








French 


2812 


1116 


39.7 


774 


69.4 


27.5 


German 


818 


310 


37.9 


225 


72.6 


27.5 


Italian 


55 


24 


43.6 


18 


75.0 


32.7 


Russian 


266 


125 


47.0 


50 


40 . 0 


18.8 


Spanish 


1864 


691 


37.1 


464 


67.1 


24.9 


TOTAL 


5815 


2266 


39.0 


1531 


67.6 


26.3 








TOTAL 








French 


5043 


2287 


45.3 


1270 


55.5 


25.2 


Geiniian 


1748 


845 


48.3 


413 


48.9 


23.6 


Italian 


108 


64 


59.3 


28 


43.8 


25.9 


Russian 


556 


331 


59.5 


105 


31.7 


18.9 


Spanish 


4178 


1900 


45.5 


968 


50.9 


23.2 


TOTAL 


11,633* 


5427 


46.7 


2784 


51.4 


23.9 



*In addition, there were 60 students listed as majoring in Romance Languages. Some 
of these may have appeared in the tested sample , but information is not available as to 
which language they were tested in. 
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Further, of the 2787 legitimate cases: 

1116 (40. OZ) took the teacher preparation sections of the MLA tests (parts 
5-7) presumably because they were enrolled In teacher preparation pro- 
grams (there were a handful of missing scores) : 

1671 (60. OZ) did not take any of the teacher preparation tests, presumably be- 
cause they were not enrolled In teacher preparation programs. 

2534 (90. 9Z) of the 2787 students took the MLAT. there being only 5 cases with 
some missing scores. 

In all, there were 2389 (85. 7Z) complete on MLA skills test scores and the tPAT ; of 
these, 980 were also con^lete on the teacher preparation sections of the MLA battery. 

An examination of the Student Questionnaires showed that there were a number of 
Individuals In the total sample who might not be regarded as "regular” cases who were 
native speakers of English and who took their beginning and Intermediate foreign language 
Instruction In regular schools. All 2880 cases were divided Into a ftmttber of types: 
three "regular" types and six "odd" types, as follows: 

"Regular" types 



1 


Student began the study of his major 

FL In "grade school" 


. 282 


10. 8Z 


2 


Student began the study of his major 

FL In secondary school 


1662 


63. 8Z 


3 


Student began the study of his major 


. 660 


25, 3Z 




Total » . . 


2604 
(90. 4Z) 


99. 9Z 


Odd" types 








4 


The first course the student listed as 
having taken In his major FL was of an 
advanced type, from which fact It might 
be presumed that he had a non-usual 
Introduction to the language . . . . 






5 


The student's experience Included Intensive 
training In his FL In military service ...... 






6 


The student's experience Included two or more 
years spent In a country where the FL Is spoken . 


. . 17 




7 


Course data were missing or sparse • 


. 66 




8 








9 










Xotdl • • • 


. 276 

(9.6Z) 





Assignment of a student to an "odd" type category was made If there was any evidence to 
support this even though he might have listed beginning courses that would otherwise have 
led to his assignment to "regular" type categories. This was done In order to remove anj 
doubt that the "regular" cases might contain cases with Irregular or unusual experiences 
In the relevant foreign language. Most of the analyses to be reported here are based 
only on "regular" cases, or upon subgroups of these. 
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Chcipter IV 



RESULTS FROM TESTS OF FOREIGN LANGUAGE ATTAINMENT: 

THE TESTED SAMPLE 



This chapter will concern only the results of the four MLA skills tests actually 
given to the sample that volunteered to take these tests. It will show the overall 
statistics, frequency distributions, and intertest correlations obtained ff^r the "regular" 
and "odd" cases, without consideration of the fact that the tested sample was undoubtedly 
biased In several ways with respect to the target national population. Chapter VI will 
present attempted corrections for these sampling biases. Furthermore, this chapter will 
give scant attention to various breakdowns of the sample that could be made, since these 
will be the concern of later chapters that investigate the association of the test scores 
with various student background and instructional variables. 

Since the MLA skills tests constitute the chief criteria used for evaluating the 
foreign language attainments of the sample. It Is well to focus attention on the general 
characteristics of the test results. Can we tell, from these characteristics, anything 
about how well the tests appear to measure foreign language attainment? Can we say any- 
thing about whether all four tests — Listening, Speaking, Reading, and Writing— measure 
the same or different aspects of achievement? These matters will be taken up In this 
chapter. 



1. D istributional statistics, all cases 



Table 4.1 shows distributional statistics for the MLA skills test scores for all 
cases In the sample ("regular" and "odd") that had complete data on these four tests, 
for 1170 cases In French, 388 ca.ses In Germai::, 102 cases In Russian, and 900 cases; In 
Spanish. No data are shown for Italian because of the small number of cases. Figures 
4.1, 4.2, 4.3, and 4.4 depict the score distributions graphically: the scores are plotted 
in accordance with the MLA-FSI equivalents established in Chapter II, in order to facili- 
tate compari.^ons across skills and across languages. 

The scores range from values that are very low— lower than the estimated "chance" 
scores for tests where such scores can be estimated— to values that are at or near the 
maximum possible scores. It is evident, however, that some of the tests have a "celling," 
at least In ter 3 of the FSI equivalents, whereby they cannot measure differences among 
the upper FSI ratings. This Is universally the case for the Listening test, which mea- 
sures only up to S-3+ In French and Russian, S-4 In Spanish, and S-44' In German. Speak- 
ing measures only up to S-4+ In German and Russian; Reading falls to cover R-5 In French 
and Russian, and Writing falls to cover R-5 In Spanish and the range from R-3*f to R-5 In 
Russian. These ceilings are also reflected to a considerable extent in the negatively 
skewed distributions for some of these tests. For example, the distribution of scores 
for French Listening seems to be sharply cut off at its maximum score, which corresponds 
to an FSI rating of S-3+. If the test had been constructed with more difficult items 
it might have been able to discriminate among the upper FSI ratings. Indeed, all the 
distributions of Listening test scores, save that in Russian, are negatively skewed to a 
highly significant extent, and so are most of the distributions of the Writing test scores. 
Although the MLA tests are billed as "advanced" tests. In many cases they are not suffi- 
ciently difficult to discriminate among really advanced levels of proficiency . This fact 
must be borne In mind as the reader examines data to be reported here. Furthermore, the 
negative skewnesses of many of the test score distributions may make means and standard 
deviations somewhat misleading. In a great many of the statistical analyses to be report- 
ed here, means and standard deviations are employed because they are statistics reported 
by the available computer programs; medlanp and Interquartile ranges would perhaps be more 
appropriate statistics to report In the case of the negatively skewed distributions. / 
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Table 4.1 

Data from MLA Skills Test Score Distributions 
(All Cases Complete on 4 MLA Skills Tests) 




Statistic 



Max. Possible Score (Form A) 
Highest Obtained Score 
Median (Estimated) 

Mean 

Lowest Obtained Score 
Chance Score 
Standard Deviation 

(Measure of Skewness) 

Max. Possible Score (Form A) 
Highest Obtained Score 
Median (Estimated) 

Mean 

Lowest Obtained Score 
Chance Score 
Standard Deviation 
gj^ (Measure of Skewness) 

Max. Possible Score (Form A) 
Highest Obtained Score 
Median (Estimated) 

Mean 

Lowest Obtained Score 
Chance Score 
Standard Deviation 
gj^ (Measure of Skewness) 

Max. Possible Score (Form A) 
Highest Obtained Score 
Median (Estimated) 

Mean 

Lowest Obtained Score 
Chance Score 
Standard Deviation 
gj^ (Measure of Skewness) 



Listening Speaking Reading 



FRENCH (N = 1170) 



GERMAN (N 



RUSSIAN (N 



SPANISH (N 



56 


125 


70 


80 


56 


115 


69 


74 


45.3995 


73.6550 


49.3089 


47.3355 


44.3051 


73.9248 


48.9991 


47.3949 


24 


39 


27 


22 


29 




32.5 


— , 


7.6219 


9.5323 


8.6354 


9.6519 


-0.4308** 


0.0849 


-0.1076 


0.0184 


388) 


56 


125 


70 


80 


56 


124 


70 


77 


46.3129 


90.2181 


53.7248 


56.0342 


44.7990 


89.7552 


52.6753 


54.3144 


24 


45 


25 


20 


29 


— 


32.5 


— 


8.6787 


14.4050 


10.0879 


13.5483 


-0.5233** 


-0.0964 


-0.3121* 


-0.3808** 


- 102) 


56 


125 


70 


80 


56 


122 


70 


80 


43.1607 


77.8631 


41.5714 


65.9171 


43.2843 


80.8431 


43.3431 


63.3922 


32 


50 


24 


35 


29 


— 


32.5 


— 


6.2141 


14.7534 


11.1536 


11.2050 


0.0597 


0.6060* 


0.4765* 


-0.6760** 


[ - 900) 
56 


125 


70 


00 


56 


122 


69 


78 


45.9926 


85.1313 


48.3385 


56.2052 


44.9722 


85.1800 


48.6044 


55.4556 


24 


46 


27 


24 


29 


— 


32.5 


— 


6.8664 


12.7835 


8.7697 


10.5109 


-0.4458** 0.0114 


0.0910 


-0.2140* 



**p < .01, *p < .05 for hypothesis of gj - 0 and normality of distribution. 
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Figure 4.1 

Frequency Distribution of Scores on MLA Skills 
Tests, French, Tested Sample 
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Figure 4.2 

Frequency Distribution of Scores on MLA Skills 
Tests, German., Tested Sample 
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Figure 4.3 

Frequency Distribution of Scores on MLA §kills 
Tests, Russian, Tested Sample 
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Figure 4.4 

Frequency Distribution of Scores on MLA Skills 
Tests, Spanish, Tested Sample 
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GeneraUy* for a negatively skewed distribution, the median will be greater than the 
mean. 



For example, in appraising the general level of attainment in Listening achieved 
by the French group, the mean score is only 44.3, corresponding to an a 

little below S-2+. The median (as estimated by a statistical formula that employs the 
mean and the measure of skewness gj) is 45.4, more than a score point higher than e 
mean and appreciably closer to the presumed FSI equivalent of S-2+. Furthermore, it can 
be seen from the distribution (Figure 4.1) that there are 

cases exceeding the FSI S-3 level, and if the test had been more difficult it is possible 
that many of those students shown as lying between S-3 and S-3+ would have scored at 
higher FSI rating equivalents. 

Taking the results at their face value, however, we find that a general character- 
istic of the tested samples is that they are much poorer in Listening and Speaking skills 
than they are in Reading and Writing. This is true for all four 

for German, where at least the Listening median is in the range of the Reading and Writing 
medians according to the FSI equivalents. The Speaking test results are particularly 
low for the French group; in interpreting this finding, it must be borne in mind however, 
that the ratine standards of the persons who judged the French speaking test results 

Jeen for so!l^“eLo« unusually severe. (n,e footnote on page 12 suggested that rating 

Standards of judges might vary from time to time.) 

The distributions of Reading and Writing scores tend to be more normal (symmetric) 
in shape and to center around FSI ratings of R-3. Even for reading and writing skills, 
however, the results may be regarded as somewhat disappointing in comparison to what might 
be expected of foreign language majors at graduation from college. If the attain^nt of 
reading and writing skill at about the FSI R-3 level ("minimal professional skill ) is 
taken to be one of the objectives of foreign language teaching at the college level, only 
about half of the seniors in our sample attained this objective, let alone such an ob- 
jective in the listening and speaking skills. 



2 . Distributional statistics , 

»rA^tilar" and "odd" type cases 

The means and standard deviations for "regular" and various "odd" type cases, in 
all languages except Italian,^ are shown in Table 4.2. "Regular" cases, as e^lained pre- 
viously, are those who began the study of their major foreign language at an introductory 
level at some point in the normal educational sequence— in elementary school, in second- 
ary school, or in college—, and whose records showed no unusual language learning 
iences. such as prolonged residence in a foreign country, that would classify them as odd 
types. Thus, the "regular" cases were intended to represent the typical native speaker 
of English who gets his foreign language training in the American educational system. 

It may be seen by comparing data in Tables 4.1 and 4.2 that the "regular" cas^ show- 
ed mean skill scores iightly lower than those given for the total tested sample. This 
is mainly because many of the "odd" types excluded from the group had means that were con- 
siderably elevated because of their unusual language learning experiences, as will be 
pointed out below. 

"Odd" type 4 cases, who listed no introductory courses in their backgrounds and who 
may therefore be presumed to have had unusual language learning experiences (e.g. learnitag 
from parents, special tutoring, etc.), did distinctly better on the tests than 
cases, on the average. There were substantial numbers of such cases in French, German 

and Spanish. 



^For information on the test performance of students of Italian, see Table 4.3 
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Table 4.2 

Distribution Statistics for MLA Skills Test Scores 
for "Regular" and "Odd" Type Cases ^ 



Regular cases 
Odd type 4 

II II 6 

II II y 

II II 8 

II II 9 

Regular cases 
Odd type 4 

II II 5 

6 

7 

8 
9 



Regular cases 
Odd type 3 

II 5 

II II 7 

II 3 

II II 9 



II 

II 






Listening 


Speaking 


Reading 


Writing 


N 


X a 


X a 


X 


a 


X a 






FRENCH 








1115 


44.12 7.63 


73.56 9.25 


48.81 


8.59 


47.16 9,54 


24 


49.08 5.71 


82.91 12.25 


54.80 


7.41 


52.40 9.06 


5 


51.60 1.87 


73.80 8.96 


52.40 


9.57 


53.20 11.87 


13-16 


44.44 9.87 


78.06 13.31 


50.46 


10.75 


49.00 12.63 


9-10 


48.30 7.52 


82.11 10.29 


52.11 


12.41 


51.90 12.47 


7 


49.43 6.71 


85.57 14.57 


56.14 


9.11 


59.43 11.55 






GERMAN 








324 


43.44 8.62 


86.75 12.80 


51.03 


9.75 


52.09 13.33 


27-29 


51.90 5.34 


104.96 11.83 


59.68 


8.39 


65.11 9.16 


5 


51.40 3.59 


97.60 14.29 


56.00 


9.75 


63.00 6.76 


4-5 


49.80 5.33 


102.50 12.53 


60.00 


7.07 


62.40 4.08 


12-15 


45.53 8.74 


95.23 15.14 


58.67 


6.22 


63.25 5.83 


2 


54.00 — 


102.50 


63.00 


— 


61.00 


15-17 


52.44 2.97 


113.73 8.14 


65.77 


4.68 


70.06 6.96 






RUSSIAN 








89 


42.53 5.97 


78.29 12.61 


41.25 


9.59 


62.07 10.84 


3 


44.00 — 


93.33 


56.67 


— 


68.331 — 


5-6 


44.-50 — 


87.00 • — 


50.00 


— - 


69.33 — 


1-2 


46.00 — 


120 


49.50 


— 


69.00 — 


1 


56.00 


115 


70 


— 


80 


3 


51.33 — 


109.33 


59.33 




70.33 — 






SPANISH 








816 


44.47 6.79 


83.99 12.01 


47.83 


8.41 


54.62 10.13 


36 


50.08 5.80 


96.97 13.08 


56.56 


7.92 


62.92 11.29 


1 


43 


82 


45 


— 


45 


3-5 


50.75 — 


104.00 


57.60 


— 


65.40 — 



college . 

"Odd" type cases were classified as follows! 

T o A. TJlrst course listed was an intermediate or advanced college-level course, 
Srricflt pre,u..d that their introduction to the language waa 

not ordinary. 

Type 5: Took training in the language during military service. 

Type 6: Had spent two or more years in a country where the foreign language is the 

normal means of communication. 

Type 7; Data on courses were missing, or so sparse mat the case could not be 
otherwise classified. 

Type 8: A miscellaneous category. 

Type 9: Evidence from the questionnaire (often in the form of 

ments) indicated that the student was a native speaker of the lang g 

which he was majoring. 
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Type 5 esses were those who indicated they had studied their major language in 
intensive courses given while they were in military service. Since*' there was no pro- 
vision on the questionnaire for stating this, these comments were volunteered. There 
were 5 such cases in German, 6 in Russian, and 1 i.%. Spanish. They^^did distinctly better 
on the skills tests, on the average, than the regular cases. This finding speaks well 
for the value of intensive language courses in military service. Of course, it could 
have happened that these students also had an opportunity to Improve their language skills, 
after training, with practical work such as interpreting and translating. 

Type 6 cases were those who volunteered the indication that they had spent two or 

more j^ears residing in a country where the foreign language is the normal means of commu- 

nication. There were 5 such cases in French, 5 in German, and 5 in Spanish. In general, 
they performed quite well on the tests, on the average, as compared to the regular cases. 

In German and Spanish, at least, their Speaking test scores were strikingly better than 
those of the regular cases. This finding agrees with a result to be discussed later in 
this report (Chapter Vlt) , namely, that time spent abroad is an Important variable in the 
attainment of foreign language skill 

Type 7 cases were those who could not be otherwise classified; for reasons unknown, 

they tended to perform somewhat better on the tests than regular cases. 

Type 8 was a miscellaneous category reserved for those indicating various types of 
unusual language learning experiences not otherwise classifiable. 

Particular Interest attaches to the results for "odd" type 9, persons for whom there 
was evidence that they were native speakers of the language in which they were majoring. 
Through oversight, there was no provision in the questionnaire for indicating this, and 
therefore some cases of native speakers may have been Included in the "regular case" cate- 
gory. Students were classified as type 9 only when there was clear evidence from the 
questionnaire, usually in the form of a volunteered statement, that they were native 
speakers. Of course, the definition of a native speaker presents some difficulties: many 

people could claim to be native speakers of a language learned in childhood and since for- 
gotten. ^ 

At any rate, the data show 7 native speakers of French, 17 of German, 3 of Russian, 
and 26 of Spanish. We would expect such individuals to perform near the top of the score 
distributions, well in the range of FSl ratings of 4, 44*, and 5. (It should be remarked 
that the FSl rating of 5 is reserved for educated native speakers. We have no Information 
on how the FSl procedure would rate native speakers with less than a university education.) 
The means for native speakers in Table 4.2 are in every case distinctly higher than those 
for regular cases; in fact, they are generally the highest among the means for the various 
"odd" type cases. The separation of the native speakers from the regular cases is particu- 
larly striking in the case of German and Spanish, where, in addition, the standard devia- 
tions tend to be much lower (indicating more homogeneity) than those of the regular cases. 
The French native speakers are, for some reason, not so distinctly superior. 

It is only in German and Spanish that the native speakers have mean scores that are 
in the range of FSl ratings of 4, 44*, and 5. In French and Russian, the scores have FSl 
equivalents that tend to be more in the range of 3 and 3+, There are several possible 
reasons for the failure of the native speaker scores to have high FSl equivalents. First, 
the procedures employed for setting up the equivalents have many drawbacks: there were 
few if any native speakers Involved in the equivalence experiment; there was in any case 
insufficient range of ratings and scores, and insufficient numbers of cases, to establish 
reliable equivalences; and the FSl rating procedure Itself undoubtedly is not perfectly 
reliable or valid. Second, many of the tests appear to have ceilings; that is, they 
are not long enough, or dlfflc-islt enough, tj differentiate among the upper levels of lan- 
guage proficiency. Third, we do not have enough information about the "native speakers" 
in our sample to know whether they would be judged as native speakers by Independent 
criteria. (Nevertheless, if a student claimed he was a native speaker of the language in 
which he was majoring, it is reasonable to assume that his knowledge of it was quite well 
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formed through recent study even if he had once forgotten the language.) Finally, there 
is no guarantee that the MLA skills tests are sufficiently reliable and valid to identify 
native speakers with certainty. 



3. Norms for Regular Cases 

For certain purposes, it may be useful to have percentile norms for all the ''regular” 
cases that were actually tested in the 1965 national study. These are presented in Table 
4.3. They Include, incidentally, information on the relatively small number of cases 
tested in Italian, and also data on the teacher preparation tests in the MLA battery (re» 
suits of which are discussed in Chapter X). For norms estimated for the national sample 
after correction for sampling fractions, see Tables 5.11, 5.12, 10.3 , and 10.4 . The 
numbers, of cases in Table 4.3 are slightly higher than those shown for regular cases in 
Table 4.2 because they include some cases whose data were otherwise incomplete. 



4. Intertest correlations 

Investigation of the correlations among the skills tests is of interest for the in- 
formation it yields as to the extent to which these tests measure the same thing— overall 
language proficiency— or different things, i.e., language proficiency as exhibited in 
specific skills. The Pearsonian product-moment correlations among the four skills, for 
French, German, Russian, and Spanish, are shown in Table 4.4. Shown also, for comparison, 
are the intercorrelations reported (Educational Testing Service^ 1961a^ 1961b^ 1962at 
1962b) for "post-test" sedres of standardization samples of NDEA Institute teachers. The 
"post— test" scores were those obtained towards the end of the NDEA Institutes, l.e., after 
the teachers had received refresher training in the foreign language. These data were 
selected for comparison with the present data because they seemed more comparable, with 
respect to mean score levels, than the pre-test data (tests administered at the outset of 
the NDEA Institutes) . For the present data, the numbers of cases are the same as those 
of Table 4.1; that is, they represent all cases ("regular" and "odd") that were complete 
on the four skills tests. 

In view of the fact that the NDEA data appear to represent, judging by the standard 
deviations, a somewhat more heterogeneous sample than our sample, the correlations for the 
CFLT sample are not inconsistent with those obtained for NDEA Institute cases. That is, 
they are in general somewhat lower, but this is to be expected when the sample is more 
homogeneous . 

All the correlations for the CFLT sample are highly significant in the sense that they 
depart significantly from what might be expected if the true correlations were zero. They 
are in fact almost uniformly high, high enough to indicate that the four skills tests 
measure, in the main, a single "common factor" of proficiency in the language concerned. 
Table 4.5 shows the proportions of test variance that can be accounted for by a single 
common factor, using a procedure described by Harman (1960, p. 122). Except in Russian, 
where the Listening test has the highest proportion of variance accounted for by a common 
factor, the Writing test is the test that has the highest proportion of common factor 
variance. It may be regarded, therefore, as the best single measure of overall language 
proficiency. The proportions of common variance in the tests range generally . from about 
.70 to as high as .88, except for the Speaking test, where the common factor variances 
range from .53 to .72, due to a lower reliability of the Speaking test; partly it may be 
due also to a greater specificity of the skill measured by the Speaking test. 

It should be noted, however, that these computations have not taken account of the 
possible effect of unreliability. Statistical procedures are available (Gulliksen, 1950, 
pp. 101-104) to correct correlations for the effects of unreliability, that is, to esti- 
mate what the correlations would be if the tests were perfectly reliable. This is often 
called the "correction for attenuation." However, it was not an objective of this in- 
vestigation to determine the reliabilities of the tests. In order to perform the correc- 
tion for attenuation, however, it is necessary to have estimates of the reliabilities of 
the tests applicable to the present sample. For this purpose we take the reliabilities 



o 
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Table 4.3 

Norms ^ for MLA. Advanced FL Proficiency Tests 
Based on "Regular" Cases Tested, 

1965 College Senior Major Samples 



Xile 

Rank 


List . 


Speak. 


99 


56 


107 


97 


55 


93 


95 


55 


90 


90 


54 


86 


85 


53 


83 


80 


52 


81 


75 


51 


80 


70 


50 


78 


65 


49 


77 


60 


47 


76 


55 


46 


75 


50 


45 


74 


45 


44 


73 


40 


43 


72 


35 


41 


70 


30 


40 


69 


25 


38 


67 


20 


37 


66 


15 


35 


64 


10 


33 


62 


5 


30 


58 


3 


29 


56 


1 . 


27 


52 



FRENCH 




Appl. 


Read. 


Writ. 


Ling. 


70 


75 


67 


66 


69 


63 


64 


64 


61 


61 


61 


58 


59 


59 


56 


57 


56 


55 


56 


55 


54 


54 


53 


52 


53 


52 


51 


52 


50 


50 


51 


49 


49 


49 


47 


48 


48 


48 


47 


47 


45 


47 


46 


43 


46 


44 


42 


45 


43 


40 


44 


41 


39 


42 


39 


37 


41 


37 


34 


39 


34 


31 


37 


33 


29 


36 


31 


26 


33 



liv. & 


Zile 




Cult. 


Rank 


List . 


67 


99 


56 


59 


97 


56 


58 


95 


55 


55 


90 


54 


53 


85 


53 


52 


80 


52. 


50 


75 


51 


49 


70 


50 


48 


65 


49 


47 


60 


48 


47 


55 


46 


46 


50 


45 


45 


45 


44 


44 


40 


43 


43 


35 


40 


43 


30 


38 


42 


25 


36 


41 


20 


34 


40 


15 


33 


38 


10 


31 


36 


5 


29 


34 


3 


28 


32 


1 


27 



GERMAN 



Speak. 


Read. 


Writ. 


Appl. 

Ling. 


Civ. & 
Cult. 


115 


69 


77 


65 


70 


110 


68 


74 


64 


67 


107 


67 


72 


62 


66 


103 


64 


69 


59 


64 


102 


61 


67 


58 


61 


98 


60 


65 


56 


59 


96 


59 


63 


55 


57 


94 


57 


61 


54 


55 


92 


56 


60 


53 


54 


91 


55 


58 


32 


53 


89 


53 


56 


51 


51 


87 


52 


54 


50 


50 


86 


50 


51 


48 


49 


84 


49 


48 


47 


48 


83 


48 


46 


46 


47 


81 


45 


44 


45 


46 


79 


44 


42 


44 


44 


76 


42 


40 


44 


43 


73 


40 


37 


42 


42 


69 


38 


34 


41 


40 


66 


34 


30 


38 


37 


62 


32 


26 


37 


37 


57 


30 


24 


36 


35 



No. 

Cases 

X 

a 



1184 

44.000 

7.666 



1123 

73.568 

9.265 



1194 

48.774 

8.614 



1193 

47,044 

9.573 



445 

48.661 

7.235 



460 
46 . 10 ^^ 
6.4 7 



338 327 

43.618 86.817 
8.583 12.808 



337 338 

51.095 52.183 
9.680 13.224 



116 111 
49.879 50.928 

7.278 8.752 



Zile 

Rank 

99 

97 

95 

90 

85 

80 

75 

70 

65 

60 

55 

50 

45 

40 

35 

30 

25 

20 

15 

10 

5 

3 

1 

No. 

Cases 

X 



lilst . 

55 

51 

48 

46 

45 

44 

43 

43 

42 

42 

42 

41 

41 

40 

40 

38 

37 

33 

32 

31 

29 

27 

24 



24 

39.750 

5.613 



ITALIAN (tentative) 



Speak. 


Read. 


Writ. 


102 


55 


68 


99 


54 


66 


97 


53 


65 


94 


52 


64 


93 


49 


63 


92 


48 


60 


91 


47 


59 


91 


46 


58 


90 


45 


57 


90 


44 


56 


89 


43 


55 


87. 


42 


54 


85 


41 


53 


84 


40 


32 


83 


40 


51 


81 


39 


50 


78 


38 


49 


76 


37 


48 


72 


37 


46 


70 


34 


42 


65 


29 


37 


62 


28 


33 


57 


27 


25 


24 


24 


23 


84.833 


42.292 


53.304 


10.893 


6.931 


8.728 



Appl. 


Civ. Cl 


Zile 


Ling. 


C'llt . 


Rank 


58 


58 


99 


57 


56 


97 


56 


:5 


95 


55 


34 


90 


54 


53 


85 


54 


52 


80 


53 


51 


75 


53 


50 


70 


52 


49 


65 


52 


49 


60 


51 


48 


55 


49 


47 


50 


47 


46 


45 


47 


44 


40 


46 


44 


35 


46 


43 


30 


46 


43 


25 


45 


42 


20 


44 


40 


15 


44 


38 


10 


43 


35 


5 


43 


32 


3 


42 


28 


1 



8 8 
49.375 46.125 

4.947 6.451 



RUSSIAN 



List. 


Speak. 


Read. 


Writ. 


Appl. 

Ling. 


Civ. & 
Cult. 


56 


112 


65 


80 


69 


78 


53 


107 


60 


78 


64 


71 


52 


101 


59 


77 


61 


64 


51 


96 


55 


75 


57 


63 


49 


89 


52 


73 


56 


62 


48 


87 


49 


72 


55 


60 


47 


85 


48 


70 


52 


57 


46 


83 


46 


69 


51 


56 


45 


81 


44 


68 


50 


55 


44 


80 


43 


67 


50 


54 


43 


79 


42 


66 


49 


53 


43 


78 


40 


65 


49 


52 


42 


77 


39 


63 


47 


52 


41 


76 


37 


61 


46 


51 


39 


75 


36 


59 


46 


50 


38 


74 


35 


56 


45 


49 


37 


72 


34 


54 


45 


4R 


36 


69 


32 


52 


44 


46 


35 


65 


30 


51 


44 


45 


34 


60 


29 


49 


43 


43 


33 


56 


27 


40 


42 


41 


33 


53 


26 


38 


40 


39 


31 


50 


24 


35 


37 


36 


92 


91 


92 


90 


18 


20 


42.304 


78.198 


41.033 


61.956 


49.111 


52.800 


6.087 


12.846 


9.676 


10.950 


6.253 


8.171 



%ile 

Rank 

99 

97 

95 

90 

85 

80 

75 

70 

65 

60 

55 

50 

45 

40 

35 

30 

25 

20 

15 

10 

5 

3 

1 



Zile 

Rank 

99 

97 

95 

90 

85 

80 

75 

70 

65 

60 

55 

50 

45 

40 

35 

30 

25 

20 

15 

10 

5 

3 

1 



O 
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Table 4.3 (continued) 



SPANISH 



%ile 



Rank 


List. 


Speak . 


Read 


99 


56 


125 


66 


97 


55 


no 


64 


95 


54 


105 


62 


90 


53 


100 


59 


85 


52 


97 


57 


80 


51 


94 


56 


75 


50 


92 


54 


70 


49 


90 


52 


65 


48 


89 


51 


60 


47 


88 


50 


55 


46 


86 


49 


50 


45 


85 


47 


45 


44 


83 


46 


40 


43 


82 


45 


35 


42 


80 


44 


30 


41 


78 


43 


25 


40 


76 


41 


20 


38 


74 


40 


15 


37 


72 


39 


10 


35 


69 


37 


5 


32 


63 


34 


3 


31 


60 


33 


1 


29 


54 


31 


No. 


Cases 


860 


821 


862 


X 


44.442 


83.962 


47.773 


a 


6.790 


12.005 


8.423 





Appl. 


Civ. & 


%ile 


Writ. 


Ling. 


Cult. 


Rank 


78 


70 


72 


99 


73 


63 


69 


97 


71 


61 


68 


95 


68 


58 


66 


90 


66 


55 


64 


85 


64 


53 


63 


80 


63 


51 


62 


75 


61 


50 


60 


70 


60 


49 


59 


65 


58 


48 


58 


60 


57 


47 


57 


55 


55 


46 


56 


50 


54 


45 


55 


45 


53 


44 


54 


40 


51 


43 


52 


35 


49 


43 


51 


30 


47 


42 


50 


25 


45 


40 


49 


20 


43 


39 


47 


15 


41 


37 


45 


10 


37 


35 


42 


5 


35 


34 


41 


3 


31 


33 


39 


1 


866 


394 


405 




54.510 


46.680 


55.536 




10.203 


7.426 


7.663 





PROFESSIONAL PREPARATION 
ALL LANGUAGES 



80 

75 

74 

72 

70 

69 

68 

67 

66 

65 

64 

63 

62 

61 

60 

59 

57 

56 

55 

53 

49 

47 

41 



991 

62.126 

7.435 



^For estimated norms for the national population, see Chapter V. 
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Table 4.4 

Intercorrelations of MLA Skills Tests for CFLT National Sample, 
as Compared to Results for NDEA Institute Standardization 
Samples (ETS, 1961a, 1961b, 1962a, 1962b) 



CFLT N 
Listening 



FRENCH 
1170 NDEA N 



Speaking 



1336* 
Reading Writing 



Listening 

Speaking 

Reading 

Writing 



1.00 

[.844] 

.68 

(.800) 

.73 

(.800) 

.75 

(.797) 



.68 

(.800) 

1.00 

1.825] 



.73 

(.800) 



.58 

(.736) 

.65 

(.782) 



.58 

(.736) 

1.00 

[.883] 

.80 

(.858) 



.75 

(.797) 

.65 

(.782) 

.80 

(.858) 

1.00 

[.924] 



CFLT N 



GERMAN 
388 NDEA N 



297* 



Listening Speaking Reading Writing 



Mean 



S.D. 



A) 


44.3 

(42.0) 


73.9 

(83.6) 


49.0 

(47.4) 


47.4 

(46.9) 




7.6 

(8.4) 


9.5 

(17.1) 


8.6 

(10.5) 


9.7 

(12.5) 



CFLT N 



RUSSIAN 

102 NDEA N = 176* 





Listening 


Speaking 


Reading 


Writing 


Listening 


1.00 

[.798] 


.69 

(.746) 


.81 

(.766) 


.78 

(.754) 


Speaking 


.69 

(.746) 


1.00 

[.786] 


.68 

(.70S) 


.68 

(.666) 


Reading 


.81 

(.766) 


.68 

(.708) 


1.00 

[.850] 


.75 

(.705) 


Writing 


.78 

(.754) 


.68 

(.666) 


.75 

(.705) 


1.00 

[.914] 



Mean 
(Form A) 


43.3 

(39.8) 


80.8 
(74. G) 


43.3 

(39.4) 


63.4 

(57.9) 


S.D. 


6.2 

(5.8) 


14.8 

(17.0) 


11.2 

(10.6) 


11.2 
(14 e 5) 



Listening 


1.00 

[.844] 


.78 

(.725) 


.85 

(.840) 


.85 

(.806) 


Speaking 


.78 

(.725) 


1.00 

[.806] 


.78 

(.700) 


.80 

(.771) 


Reading 


.85 

(.840) 


.78 

(.700) 


1.00 

[.900] 


.87 

(.864) 


Wilting 


.85 

(.806) 


.80 

(.771) 


.87 

(.864) 


1.00 

[.924] 


Mean 
(Form A) 


44.8 

(43.0) 


89.8 

(85.5) 


52.7 

(48.7) 


54.3 

(51.3) 


S.D. 


8.6 

(7.0) 


14.4 

(12.9) 


10.1 

(11.1) 


13.5 

(16.4) 






SPANISH 






CFLT N 


= 900 NDEA N = 1334* 




Listening 


Speaking 


Reading 


Writing 


Listening 


1.00 

[.860] 


.67 

(.770) 


.77 

(.800) 


.77 

(.803) 


Speaking 


.67 

(.770) 


1.00 

[.789] 


.61 

(.702) 


.65 

(.724) 


Reading 


.77 

(.800) 


.61 

(.702) 


1.00 

[.877] 


.81 

(.862) 


Writing 


.77 

(.803) 


.65 

(.724) 


.81 

(.862) 


1.00 

[.902] 


Mean 
(Form A) 


45.0 

(42.4) 


85.2 

(79.8) 


48.6 

(44.5) 


55.5 

(49.2) 


S.D. 


6.9 

(7.8) 


12.8 

(16.6) 


8.8 

(9.7) 


10.5 

(12.6) 



.Forms A sod B «« KS'rdlesrof'^l-^li." S^rriaS^ns 

irxSerr sirs: rs. 

rs tSSrlSrSrTSnSe: r crarSnSS. .^ 0 , reamed mere comFaraBle rhae pre- 
test data to the CFLT data. 
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Table 4«5 



Proportions of Common Factor Variance 
Accounted for by the MLA Skills Tests’^ 

(Derived from Correlations for the CFLT Sample In Table 4.4) 



Test 


FRISNCH 


GERMAN 


RUSSIAN 


SPANISH 




Listening 


.7655 


.8361 


.8194 


.7849 




Speaking 


.5322 


.7223 


.5986 


.5280 




Reading 


.7074 


.8564 


.7760 


.7596 




Writing 


.8078 


.8778 


.7456 


.8052 




*See Table 4.6 


for estimates after correction for attenuation. 





of the tests given for the NDEA standardization sample (Educational Testing Service, 1964) 
and estimate reliabilities for the present sample by taking account of the differences In 
standard deviations. The results, using a formula given by Gulllksen (1950, p.lll, form- 
ula 5), are shown In Table 4.6, which also shows correlations corrected for attenuitlon 
ffti/i proportions of variance attributable to a common factor after co.rrection for attenu- 
ation. Data from the NDEA standardization sample do not show reliabilities foi any of the 
Speaking test^» but the leaflet published by Educational Testing Service Indicates that In 
certain experiments (with what languages we are not told) an Interscorer reliability of 
.89 was obtained for the Speaking test. This figure, therefore, was used for all Speaking 
tests In the present computations. 

Several points are of note concerning Table 4.6. First, the generally somewhat lower 
reliability coefficients for the CFLT sample, as compared to those given for the NDEA In- 
stitute standardization samples, are evidently due to the generally smaller variance of 
the CFLT sample. The reliability of the French Speaking test Is estimated as only .64 for 
the CFLT sample. (Of course, the standard error of measurement Is assumed to be constant 
over samples.) This explains the somewhat depressed correlations for the Speaking test In 
Table 4.4. After tie correlations are corrected for attenuation, as shown In Table 4.6, 

It would appear that the various skills tests are even more strongly associated; that Is, 
they are shown even more to measure a single common factor of language proficiency. The 
estimated proportions of common factor variance shown In Table 4.6 range from .7042 to 
.9338. For all languages except Russian, the Writing test maintains Its superiority as 
the best measure of that common factor, and except In French, the Speaking test Is least 
associated with the common factor. 

It Is not surpri. Ing that the four skills tests should be found to measure primarily 
a single factor of language proficiency In common. Basic competence In a language— know- 
ledge of its phonology, morphi‘»logy, syntax, and lexicon— is required by each of the tests, 
no matter what particular "skill” it measures. The high loading of the Writing test on 
the common factor may reflect the fact that this test Is probably most demanding with 
respect to the morphology and syntax of the language. Many of the other tests appear to 
demand knowledge primarily of lexicon, which some would regard as less close to the heart 
of language structure. 

The fact that the Speaking test Is least associated with the common factor of overall 
language proficiency may indicate that the requirements of the task set by this test are 
fairly specific and possibly to some extent unrelated to the measurement of language pro- 
ficiency. The examinee may be unaccustomed to speaking Into a microphone and his perform- 
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Table 4.6 



Estimated Reliability Coefficients for MLA Tests Applied to 
CFLT Sample and Estimated Correlations 
Corrected for Attenuation (Derived from Data of Table 4.4) 





NDEA 

Relia- 

bility 


Est. *d 
Relia- 
bility, 
CFLT 


Correlations Corrected 
for Attenuation 


Est.'d Proportions 
of Common 

Factor Variance After 
Attenuation Correction 


List • 


Speak. 


Read. 


Writ. 










FRENCH 








Listening 


.91 


.89 


(1.00) 


.90 


.81 


.83 


.8601 


Speaking 


.89* 


.64 


.90 


(1.00) 


.76 


.85 


.8316 


Reading 


.93 


.90 


.81 


.76 


(1.00) 


.88 


.7860 


Writing 


.94 


.90 


.83 


.85 


.88 


(1.00) 


.8975 










GERMAN 








Listening 


.90 


.93 


(1.00) 


.85 


.92 


.91 


.8990 


Speaking 


.89* 


.91 


.85 


(1.00) 


.85 


.87 


.7937 


Reading 


.94 


.93 


.92 


.85 


(1.00) 


.93 


.9209 


Writing 


.97 


.94 


.91 


.87 


.93 


(1.00) 


.9338 










RUSSIAN 








Listening 


.86 


.88 


(1.00) 


.80 


.90 


.87 


.9311 


Speaking 


.89* 


.85 


.80 


(1.00) 


.76 


.76 


.7042 


Reading 


.92 


.93 


.90 


.76 


(1.00) 


.81 


.8344 


Writing 


.96 


,93 


.87 


.76 


.81 


(1.00) 


.8017 










SPANISH 








Listening 


.91 


.89 


(1.00) 


.78 


.87 


.86 


.8819 


Speaking 


.89* 


.82 


.78 


(1.00) 


.71 


.75 


.6439 


Reading 


.91 


,89 


.87 


.71 


(1.00) 


.90 


.8535 


Writing 


.94 


.91 


.86 


.75 


.90 


(1.00) 


.8848 



*Assumed reliability for all tests on the basis of a report of interscorer 
reliability found in an experiment done at ETS (ETS, 1964a, p. 30). 
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ance nay depend not so much on his language proficiency as on his general fluency and 
self-confidence in spontaneoiBS speech— vhether in his native language or a foreign lang- 
uage. 

Linearity of intertest regressions 

Each Pearsonian product-moment correlation coefficient shown in Table 4.4 is actually 
the slope of the straight line of best fit to the regression of one variable on another 
variable (both variables being in standardized form). Interpretation of such correlation 
coefficients must be qualified by the assumption that a strai^^ht line is (aside from 
sampling fluctuations) the best fit to the data. Often • a curved line is a significantly 
better fit to the regression, in which case prediction of one variable from another should 
take account of this. Furthermore, curvilinear regressions are often an indication that a 
given test does not have a uniformly spaced scale of measurement; for example, that it has 
a "ceiling" or a "floor." We have already adduced evidence that some of the MLA skills 
tests have ceilings, in that they fail to measure adequately the upper levels of achieve- 
ment. A finding of a curvilinear relationship between test variables could also indicate 
some specially interesting functional relation between the underlying variables. 

Out of the 48 possible combinations of MLA skills tests language by language, there 
were only 6 instances of significant departures of regressions from linearity, four sig- 
nificant at the IZ level and 2 others significant at the 5Z level; these instances are 
detailed in Table 4.7. Although one or two of these instances may be significant only by 
chance, they are all worth examining. The best way of doing this is to plot the regres- 
sion lines graphically, end consequently we present Figures 4.5 through 4.9. 

Some explanation is due<. Each figure presents a plot of the means for the ordinate 
variable (the variable on the vertical scale) for given values of the abscissal variable 
(the variable on the horizontal scale). For example, in Figure 4.5, the French Listening 
score means are plotted against given values for the French Speaking test score. The 
point at the left lower extreme indicates that the average value of the French Listening 
score is 33.4 for all cases with scores oh the French Speaking test grouped near 50.7; 
the next point indicates a mean Listening score of 34.0 for cases grouped near 57.5; and 
so on. The points do not form a straight line, as they would, approximately, if the re- 
gression were linear. 

Figure 4.5 suggests that measurement on the French listening test is attenuated at 
both extremes its distribution; that is, on the assumption that there is valid meas- 
urement at the extremes of the Speaking locale, such measurement is not well reflected in 
the Listening test scores. The regression line tends to flatten out both at the bottom 
and the top of the curve. At the bottom, it approaches a chancre score, and at the top, 
it approaches the maximum score. Thus, we may conclude that the French listening test 
has an insufficient range of difficulty for the target population: it does not provide 

enough "floor" for low ability students, or enough "ceiling" for the students of advanced 
levels of achievement. This conclusion is supported also by the results depicted in 
Figure 4.6, where it is seen that there is some bending of the regression line as it ap- 
proaches the maximum possible Listening score. There appears to be no significant bending 
at the bottom, to be sure, but it is possible that the lower end of the Writing test is 
also insufficiently discriminative. 

A similar phenomenon is to be observed for the Listening test results in German, 
Russian, and Spanish in Figures 4.7, 4.8, and 4.9, respectively. In Figure 4.7, for Ger- 
man, we again may make the assumption that the Speaking test score is adequately discrim- 
inative at the extremes of its distribution, and on this basis we conclude that the German 
Listening test has a low celling whereby students at advanced ler^ls are not adequately 
discriminated. There is also some suggestion that the German Listening test does not 
measure adequately at the lower end of the scale, either. In Figure 4.8, for Russian, the 
ordinates are reversed: for some reason, the significant departure from linearity is 

observed for the regression of Speaking on Listening rather than for the regression of 
Listening on Speaking, as was the case in French and German. Nevertheless, the net effect 
is the same: large differences at the upper end of the Speaking test score fail to be 




FRENCH LISTENING SCORE 



r 
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J 



GERMAN SPEAKING SCORE 



57 
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SPANISH LISTENING SCOEE 
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ref lected in correspondingly-sized differences on the Listening test, suggesting that the 
Rufjsian Listening teat is not sufficiently difficult. In Spanish, the low ceiling of the 
Listening test eadilblts Itself when the regression of the Reading test on Listening is ex- 
amined • At the same time the flattening of the regression line at the lower end suggests 
that the Spanish Reading test does not have enough "floor" to accommodate, low ability 
students who are, nevertheless, differentiated by the Listening test. 

Figure 4.7 suggests also that the German Writing test may not have enough celling to 
differentiate advanced students who are, nevertheless, differentiated by the Speaking test. 

The results here are entirely consistent with the observations made in Section 1 of 
this chapter that the Listening tests in all languages fail to cover enough territory on 
the FSI scale. It will be recalled from that Section that Speaking tests were generally 
adequate in their coverage of the FSI scale, however, and this fact tends to justify the 
assumption that has been made here about the Speaking tests, namely, that they do provide 
adequate differentiation among students at either extreme of the scale. 

The observations made in this chapter concerning test characteristics could well be 
useful in preparing revised or further forms of the NLA skills tests. 



Table 4.7 

Instances of Significant Departure from Linearity 
in Intercorrelations of MLA Skills Tests 

Independent Dependent (Eta) 



Language 


N 


Variable 


Variable 


r 


n 


F-ratlo 


FRENCH 


1170 


Speaking 


Listening 


.68 


.69 


4.9** 






Writing 


Listening 


.75 


.76 


3.2** 


GERMAN 


388 


Speaking 


Listening 


.78 


• 

00 

o 


4.6** 






Speaking 


Writing 


.80 


.81 


2.5* 


RUSSIAN 


102 


Listening 


Speaking 


.69 


.74 


2.7* 


SPANISH 


900 


Listening 


Reading 


.77 


.79 


4.8** 



{ 



**p < .01; *p < .05 
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Chapter V 



ESTIMATES OF FOREIGN LANGUAGE SKILL ATTAINMENTS IN THE 

TOTAL POPULATION 



The previous chapter reported results actually obtained from the stratified sample 
of college senior foreign language majors obtained through a nationwide testing program. 
It was pointed out, however, that the sample may have contained some bias due to non- 
response of institutions and students. This chapter attempts to estimate the amount of 
this bias, at least that due to student non-response. Furthermore, it presents national 
norma estimated by a process of correcting for variation in the representation of the 
different strata in the sample. 



1. Bias due to student non-response 



We take up the matter of student non-response first because any corrections for it 
would have to be made prior to corrections for variations in the representation of 
strata. Student non-response is defined as the failure of a graduating FL major at a 
selected institution to volunteer to take any of the tests. 

It was pointed out in Chapter III, Section 2 that student response rates were far 
short of the desired lOOE that had been planned for in the sample design. Student non- 
response was particularly poor in the larger institutions, but it was nowhere near per- 
fect in the smaller institutions. There were few institutions where lOOZ of the avail- 
able students showed up for the testing. 

Student, non-response would not be a matter of any great concern in interpreting 
national norms if it could be shown that it was not correlated with test performance. 

That is, if tested and non-tested students do not differ in average test performance, the 
statistics obtained for the tested students would be good estimates of the statistics for 
all students — i.e., the tested pooled with the non-tested. It was decided to make a 
special study to determine whether the students actually tested could be considered to be 
a representative sample of all graduating language majors. On intuitive grounds, there 
was the possibility that the "better" students were more likely to volunteer for testing 
than "poorer" students. 

But how were the "betteiV students to be identified independently of the test res- 
ults? The only possibility that suggested itself as feasible was the use of foreign 
language grades, available on college transcripts, as independent measures of foreign 
language competence. Although such a measure was probably the best available under the 
circumstances, it cannot by any means be regarded as an adequately valid and reliable 
measure. College grades are notoriously unreliable and subject to shifting standards. 
Furthermore, they do not necessarily reflect to any great extent the foreign language 
competence levels presttmably measured by the skills tests in the MLA Proficiency Test 
battery. Grades in foreign language literature courses, for example, may have little to 
do with any aspect of the student's competence in the foreign language; they may reflect 
his sensitivity to literature, his general ability to write critical essays (whether in 
English or the foreign language), the amount of effort he puts in study, or his ability 
to impress the instructor. For many of the students in our project, the only grades in 
foreign language available would be those in literature and other courses not directly 
addressed to the promotion of foreign language skills. 

It was decided, nevertheless, to obtain the transcripts of all class of 1965 foreign 
language majors at selected institutions in order to see whether tested students tended 
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to be those with higher foreign language grades than students who did not volunteer to 

through certain statistical procedures, to estimate the amount of bias in the test score 
Statistics due to student non-response e 

Vr- this ouroose. 15 mediutn-large institutions (7 public, 8 private) that had re- 
sorted frOT llt^U gr^uatlng majora in all languages and that tested “Inhere from 25 
p«^eni ot St Jel!trwere selected (hereafter, this is called the ••bias study 
samnle") They were representative of all parts of the country and of many types of 
witution^ Brcor^spondence and/or telephone, registrars of these institutims were 
Asked to send transcripts of all students who had been enrolled as foreign language 
malors in the class of 1965. Confidentiality of the individual transcripts was promis d, 
“dTll regist';" agreed to provide them. (Most registrars charged a fee for this 

service.) 

On receipt of the transcripts, the project staff competed a 
the r^evant foreign language courses for each student. (That is, only the courses in 

S: nidenfs majo? language were counted; in the case »^Ve'I^I"‘tes*trr llttfr 
CFLT project, this was also the language in which they took the ^ Jf lU 

erades were converted to numerical codes as follows: A»5, B»4, , , 

Vl^s ^rminuses were ignored since many schools used only letters with no pluses and 

minuses attached. 

A oreliminary statistical analysis was performed to see whether there were 

c«.t Wffei«ces^etween the foreign language GPA^s of tested “J„Xl!en«irallf dlfier- 
_ - •In Tahiti 5 1 Over all schools, there was a highly significant oxiier 

ence^favoring the tested students. This result is probably rellAle even though grading 
standards may have varied from institution to institution, for the trend 
:e“lS stuSSts in all but two schools. The latter two schools were ' 

est percentages of tested students; there may be some meaning in the fact that the 
Mrticioatiag students at these institutions were on the average the best students. In 
6 nf the 13 institutions where the GFA*s favored the tested students, the differenc 
Lll SeniLc^t ^ t^S per cent level or beyond. One may conclude that there was a 
definite trend whereby the students who volunteered to take the ^®®^® 

better students, and this trend probably existed over the whole sample of institutions 
participating in the nationwide study. 

It may be noted, incidentally, that the average GPA^s for tested and nm-tested 
students c«respond to letter grades in the A to B range in nearly all cases. 

ui,.ther the trend noted above actually constituted a bias in the results of the 
MLA t«« wL lesraearhwever. To invLtlgate this question, correlative betw^ 
^ te^t .rres and foreign language grad^ were studied for f ® 

^st however, all CPA^s within institutions were converted to 

all sludents within an institution would have, on the averse tte ^ 

srifi on:rnrr?ts“' d^^ra eir^oiiirat?:^. arrura. poss. 
" rt^rfasr - 

ril - r. as®-®Yhat t*>®„“^‘®- orSlTaU^^^^ o“* 

M“%”riissir04.8rrf 2"). .nd 101 in Sp»ish (50.5% of 200>; Jhe« were no c«es 
4 in these 15 institutions. In each language group, correlations were ^ 

Snerie”..:; S:*les't and the ®®«»^®raz^GPA's "it^ -ults « ®H- 1^1® 

zt :. :firircresi“p;:n:r36 io »”«> • 

were based on only 103 casein xn rrenc , restricted and the average GPA was 

Sion.;ri?Si;^h:%or%i“^^^ HLA sSll score, and GPA^s were all posi- 
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Table 5.1 



Comparison of Foreign Lar juage GPA's of Tested 
and Non-Tested Students at Fifteen Selected Institutions, 
In Order of Percentage Tested 



Institution 

Code 


Tested 


N's 

Not 

Tested 


Total 


% 

Tested 


Tested 


Not 

Tested 


All 


_t 


P 


035 


9 


27 


36 


25 


4.174 


4.024 


4.062 


0.8793 


n.s. 


610 


10 


27 


37 


27 


4.765 


4.222 


4.369 


2.1818 


< .05 


665 


13 


31 


44 


30 


4.332 


3.949 


4.062 


2.5275 


< .05 


685 


23 


51 


74 


31 


4.345 


4.069 


4.155 


2.0836 


< .05 


860 


9 


16 


25 


36 


4.711 


4.231 


4.438 


2.1583 


< .05 


205 


15 


25 


40 


38 


4.305 


4.167 


4.219 


0.9819 


n.s. 


620 


14 


22 


36 


39 


4.346 


4,119 


4.207 


1.9749 


n.s. 


385 


11 


15 


26 


42 


4.366 


4.142 


4.237 


0.9844 


n.s. 


730 


13 


14 


27 


48 


4.290 


3.850 


4.062 


2.0575 


n.s. 


457 


23 


22 


45 


51 


4.123 


4.049 


4.087 


0.4593 


n.s. 


655 


15 


8 


23 


65 


4.387 


3.827 


4.192 


2.8351 


< .01 


287 


29 


13 


42 


69 


4.126 


3.964 


4.076 


1.1057 


n.s. 


755 


17 


4 


21 


81 


4.254 


3.602 


4.130 


2.6737 


< .05 


422 


37 


5 


42 


88 


4.284 


4.344 


4.291 


-0.2896 


n.s. 


247 


35 


4 


39 


90 


4.192 


4.587 


4.233 


-1.6067 


n.s. 


All 


273 


284 


557 


49 


4.289 


4.092 


4.189 


4.7369 


<.001 
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tive, r&nglng from *27 to »55 with a median at .S?* Since the range of actual GPA*s was 
relatively small, these correlations probably suggest that there is indeed a substantial 
relationship between foreign language GPA and ability as measured by the MLA skills tests. 
The correlations were judged large enough (except in Russian) to justify further statis- 
tical computations to estimate the amount of bias in MLA test results introduced by the 
less-than-perfect student response rate. 

In these computations, tise was made of certain theoretical relationships between 
correlational data obtained on two samples with different dispersions of scores on two 
correlated variables. These relationships are often used in "correcting" correlations 
for "restriction of range of talent." In the present case, however, they were used not 
to correct the correlations but to. estimate the means and standard deviations of MLA 
test scores for all cases if scores had been available for non-tested students as well as 
tested students. The basic assumption made in these computations Is that when a sample 
is "explicitly" selected from a larger sample in such a way that the standard deviation 
of a certain variable X in the smaller sample (Sjj) is different from the standard devi- 
ation in the larger sample (Sx> . this will not change (except for sampling fluctuations, 
which are Ignored) the parameters of the linear regression equation for predicting values 
of a correlated variable Y; that is, regardless of whether values for the larger or 
smaller sample are used. 



where Y is the predicted value of Y; Y and X are, respectively, the means of Y and X; 

Kxi and r^^ are respectively the correlations of X and Y in the larger and the smaller 
sample; and Sy and Sy are, respectively, the standard deviations of Y in the larger and 
the smaller sample. 

In the present case, the Grade-Point-Average (GPA) is the variable subject to "ex- 
plicit" selection (that is, the better students volunteered to take the tests) and may be 
designated X, while the MLA skill test score is the variable subject to "incidental" 
selection through its correlation with X, and may be designated Y. 

From the data available from the bias study sample, we know, in any given case, X, 

X (the mean GPA in the tested group), Sx, s^, r^y, 7 (the mean MLA skill test score in 
rested group), and Sy. The problem is to estimate Y and Sy, the mean and standard 
deviation of Y if all students had been tested. 

Now, y can be regarded as the value of Y that would be predicted by the regression 
line from x as a selected value of X, since it is the mean MLA skill score for the tested 
group. Rewriting the regression equation (using the values known for the tested group), 
we have: 



in whose right-hand side all values are known. 

To estimate Sy, we can use formula (20) given by Gulllksen (1950, p. 138), whose 
notation is used here: 



Tf - V - Rar sj « - ■ 





Sy 



and we can immediately solve for Y by the equation 

- ®y ^ 



Sy - 8y / 1 - rjy + rjy (5^8^) . 
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It is also possible to estimate R^^y* the correlation between MLA skill score and GPA 
In the larger saiiple* by Gulllksen's formula (18): 



Rxy - 



Sxtxy 



/ e2-2 



■I* 

^ “x^xy 



However* this Is of little Interest In the present case. 

By use of the data In Table 5.2, the estimated means and standard deviations of MLA 
skill test scores for the complete comparability samples In French, German, and Spanish 
were computed by the above formulas; the results are shown In Table 5.3. The table Indi- 
cates also: the amount of change In the means effected by the estimation procedure; the 
means and standard deviations of all "regular cases" actually tested In the national 
sample (date from Table 4.3 ); and means and standard deviations of the national sample 
adjusted for student non-response bias by shifting the means by the amount of change 
found In the comparability study and Inflating (or deflating) the standard deviations by 
the ratio of the two S.D.*s In the comparability study. What Is of chief Interest here 
Is the fact that the amounts of change (downward) effected by the estimation procedure 
are almost all less than one converted score point, even for variables with relatively 
large standard deviations. Frota this we can conclude that the amount of bias In our data 
due to student non-response Is relatively small. For practical purposes, any national 
norms derived from the data on tested students can be corrected for student non-response 
bias very easily by simply shifting converted scores by one point, downward. 

One caution here, of course. Is that the correction procedure used here depends upon 
the correlations between MLA skill test scores and foreign language grade-point -averages . 
If grade-point-averages had correlated higher than they did with MLA skill test scores, 
the corrections for student non-response would have been greater in magnitude. If d is 
the amount of shift actually computed, the maximum possible shift that could have occurred 
If the MLA-GPA correlations had been perfect would be d/r^y* 

It will be noticed in Table 5.3 that the bias study sample, even when examined in 
terms of estimated MLA skills test means for the total sample, was in nearly every case 
superior to the national CFLT sample. This probably reflects the fact that the bias 
study sample was assembled from, data from the larger institutions; it will be shown later 
(Chapter V, Section 2) that students at the larger Institutions tend to have somewhat 
higher MLA skills test scores than those at the smaller Institutions. 



f 
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Table 5.2 



Correlations among MLA Skills Test Scores and between MLA Skills Test 



Scores and Standardized FL Grade-Folnt-Average , 
bv Language, for Cases with Complete Test Scores 

(MLA-GPA Bias Study Sample at 15 Selected Institutions) 





MLA Skill Test Scores 
L S R 


Standardized 
G P A*s 

W (Mean*10, S.D.-l) 


Standardized GPA's 
All All 

tested cases 










FRENCH 


(N - 105) 


(N-124) 


(N-251) 


Mean 


45.99 


76.11 


51.88 


49.88 


10.13 


10.20 


9.94 


S.D. 


7.53 


9.11 


8.02 


9.12 


.97 


.94 


1.02 


L 


1.00 


.68 


.70 


.73 


.41 


- 


- , 


r’s S 


.68 


1.00 


.55 


.71 


.41 


- 


— 


R 


.70 


.55 


1.00 


.75 


.51 


- 


- 


W 


.73 


.71 


.75 


1.00 


.47 


— 












GERMAN (N - 36) 


(N-40) 


(N-83) 


Mean 


45.86 


89.89 


54.97 


57.92 


10.52 


10.50 


10.27 


S.D. 


8.89 


13.46 


10.10 


13.36 


.90 


.90 


.96 


L 


1.00 


.76 


.88 


.85 


.27 


- 


- 


r*s S 


.76 


1.00 


.77 


.79 


.29 


- 


— 


R 


.88 


.77 


1.00 


.90 


.34 


- 


- 


W 


.85 


.79 


.90 


1.00 


.33 


— 


— 










RUSSIAN 


(N-8) 


(N-8) 


(N-23) 


Mean 


44.63 


81.50 


43.00 


68.50 


10.99 


10.99 


10.68 


S.D. 


4.47 


8.33 


5.90 


5.71 


.69 


.69 


1.15 


L 


1.00 


.11 


.41 


.70 


-.23 


- 


- 


r*8 S 


.11 


1.00 


-.08 


.23 


-.25 


- 


— 


R 


.41 


-.08 


1.00 


.70 


-.25 


- 


— 


W 


.70 


.23 


.70 


1.00 


-.18 


— 


— 










SPANISH 


(N-84) 


(N-101) 


(N-200) 


Mean 


45.57 


89.01 


48.02 


55.81 


10.03 


10.04 


9.88 


S .D. 


6.57 


9.56 


8.36 


8.60 


.95 


.96 


.94 


L 


1.00 


.53 


.77 


.62 


.27 


- 


- 


S 


.53 


1.00 


.40 


.46 


«31 


- 


— 


R 


.77 


.40 


1.00 


.76 


.39 


- 


— 


W 


.62 


.46 


.76 


1.00 


.55 


- 


— 
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Table 5.3 

Study of Bias Due to Student Non-Response, and Estimates of 
Corrections for National Sample Data, 
for French, German, and Spanish Students 







Bias Study Samnle 




National CFLT Sample 


(Reeular Cases) 


MLA 

Skill 

Test 


Tested 

students 


Estimated 

for 

Total Sample 


Shift 

in 


Actual Data 




Corrected 

for 

Non-Response (Est . ) 


Mean 


S.D. 


Mean 


S.D. 


Mean 


N 


Mean S 


;.D. 


Mean 


S.D. 


Listen. 


(N - 
45.99 


105) 

7.53 


(N - 
45.39 


251) 

7.59 


FRENCH 

-.60 


1184 


44.00 


7.67 


43.40 


7.73 


Speak. 


76.11 


9.11 


75.39 


9.19 


-.72 


1123 


73.57 


9.27 


72.85 


9.35 


Reading 


51.88 


8.02 


51.09 


8.12 


-.79 


1194 


48.77 


8.61 


47.98 


8.71 


Writing 


49.88 


9.12 


49.05 


9.22 


-.83 


1193 


47.04 


9.57 


46.21 


9.68 











GEf(MAN 












(r - 


36) 


(N - 


83) 














Listen. 45.86 


8.89 


45.20 


8.93 


-.66 


338 


43.62 


8.53 


42.96 


8.61 


Speak. 84.89 


13.46 


88.82 


13.54 


-1.07 


32? 


86.82 


12.81 


85.75 


12.89 


Reading 54.97 


10.10 


54.03 


10.18 


-.94 


337 


51.10 


9.68 


50.16 

$ 


9.76 


Writing 57.92 


13.36 


56.71 


13.45 


-1.21 


338 


52.18 


13.22 


50.96 


13.31 











SPANISH 












(N - 


84) 


(N - 


200) 














Listen. 45.57 


6.57 


45.29 


6.57 


-.28 


860 


44.44 


6.79 


44.16 


6.79 


Speak. 89.01 


9.56 


88,55 


9.55 


-.46 


821 


83.96 


12.01 


83.50 


12.00 


Reading 48.02 


8.36 


47.51 


8.35 


-.51 


862 


47.77 


8.42 


47.26 


8.41 


Writing 55.81 


8.60 


55.07 


8.58 


-.74 


866 


54.51 


10.20 


53.77 


10.18 
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2, Variations In Foreign Language Skill 
Associated with Stratification Variables 
In the Sample Design 



Implicit in the sample design was the assumption that the mean achievement of stu- 
dents might vary with institution size and institution type (public vs. private); a 
stratified sample was used in order to control any such effects » without necessarily pre- 
supposing them. It was of interest, therefore, to see whether foreign language achieve- 
ment was Indeed associated with the stratification variables, and if so, in what ways. 

Table 5.4 shows the mean MLA skills test scores, for students of French, German, 
and Spanish classified by institution size and type. "Institution size" was defined as 
it had been in the sample design Itself, i.e. in terms of the total number of foreign 
language majors graduating in 1963 according to U. S. Office of Education figures. Only 
"regular" cases with complete sets of MLA skills test scores were employed in the tabula- 
tion. The number of students in Russian was too small to warrant analysis according to 
this scheme. Significance tests are given for the "main effects" in a two-way analysis 
of variance with unequal mnnbers in cells: the stratvm effect, the institution type 

effects, and the interaction between stratum and institution type. The stratum and in- 
gtitution type effects concern whether there are overall differences, respectively, among 
strata and between types. The interaction effect concerns whether similar stratum 
differences are to be observed in both public and private institutions (or, what amounts 
to the same thing, whether similar institution type differences are to be observed in 
sll strata) V For the stratum and institution type effects, a "significant" effect is to 
be interpreted as indicating significant differences among strata or between types. For 
the interaction effect, a significant effect means that differences in one of the varia- 
bles are not identical in each classification of the other variable. 

Institution size (i.e., stratum in the sampling design) is a highly significant 
effect in all the language groups and for all four of the MLA skills tests. The overall 
trend is for the students at the larger institutions to obtain higher mean scores. We 
will discuss the possible reasons for this after reviewing more data. 

Institution type is a very highly significant effect in French, a generally highly 
significant effect in Spanish (the exception being the data for the Speaking test), and 
a somewhat less significanc effect in German. In the German group, public and private 
institutions do not differ, overall, in mean Listening and Speaking scores, but students 
at the private institutions do tend to be superior in Reading and Writing. The general 
trend is for the private institutions to have students with higher average achievement 
than the public institutions do. 

Small but significant interaction effects are to be observed in the data for the 
German and Spanish groups, but not for the French group. This effect apparently reflects 
a trend. in both German and Spanish whereby the largest (stratum 5) private institutions 
exhibit higher mean scores, relatively, than they do at public institutions, while the 
public and private institutions differ only very liitle in the strata (1 through 4) con- 
taining the smaller institutions. Indeed, this interaction effect, in German and Spanish, 
goes far toward accounting for all the differences between institution types. The result 
be understood most easily if it is couched in the following terms: "if you want to 
find students with high achievement, go to the largest institutions, especially the 
private ones . " 

Possible Explanations for Variations among Strata and between Institution Types 

Section 1 of this chapter showed that students who volunteered to take the tests 
tended to be slightly better students, on the average, than those who failed to volunteer 
for the tests. Since student response rate tended to be smaller at the larger institu- 
tions, this fact might account for some of the tendency for larger institutions to exhibit 
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Table 5.4a 



Mean MLA Skills Test Scores, By Stratum 
and Institution Type 

FRENCH 



Stratum 


Size as 
of '62-' 63^ 


N 




MLA Skills Test Scores 




Listening 


Speaking 


Reading 


Writing 






PUBLIC INSTITUTIONS 






5 


40+ 


203 


44.13 


75.62 


48.60 


47.74 


4 


20-39 


67 


42.96 


74.13 


48.21 


47.13 


3 


10-19 


61 


40.52 


69.26 


44.70 


42.51 


2 


5-9 


51 


40.41 


70.96 


44.69 


43.78 


1 


1-4 


27 


37.37 


67.52 


40.37 


38.63 


Mean of Means, Total 


409 


41.08 


71.50 


45.31 


43.96 






PRIVATE INSTITUTIONS 






5 


40+ 


111 


47.76 


77.24 


53.32 


52.11 


4 


20-39 


188 


47.32 


75.64 


51.86 


49.74 


3 


10t19 


198 


45.19 


73.08 


49.87 


48.24 


2 


5-9 


111 


42.82 


71.89 


46.94 


44.86 


1 


1-4 


84 


38.77 


68.60 


44.85 


41.64 


Mean of Means, Total 


692 


44.37 


73.29 


49.37 


47.32 






ALL INSTITUTIONS (Means of Means) 


• 




5 


40+ 


314 


45.94 


76.43 


50.96 


49.92 


4 


20-39 


255 


45.14 


74.88 


50.04 


48.43 


3 


10-19 


259 


42.85 


71.17 


47.28 


45.37 


2 


5-9 


162 


41.61 


71.42 


45.82 


44.32 


1 


1-4 


111 


38.07 


68. 06 


42.61 


40.13 


Mean of Means, Total 


1101 


42.72 


72.39 


47.34 


45.64 





SIGNIFICANCE TESTS 


(F-ratlos) 






Effect 


ndf s 










Stratum 


4, 1091 


29.3** 


22.3** 


21.8** 


23.9 


Type 


1, 1091 


58.0** 


10.2** 


59.2** 


35.9** 


Stratum x Type 


4, 1091 


1.1 


0.7 


0.8 


1.6 



^Total number of FL majors 
**p < ,01 *p < .05 
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Table 5.4b 

Mean MLA Skills Test Scores, By Stratum 
and Institution Type 

GERMAN 



Stratum 


Size as 
of '62-'63^ 


N 




MLA Skills 


Test Scores 




Listening 


Speaking 


Reading 


Writing 






PUBLIC INSTITUTIONS 






5 


40+ 


75 


44.20 


86.77 


51.20 


53.97 


4 


20-39 


22 


42.73 


87.41 


50.59 


49.45 


3 


10-19 


24 


45.33 


88.62 


51.71 


53.08 


2 


5-9 1 


14 


37.07 


82.86 


45.00 


43.07 


1 


1-4 J 






‘ 






Mean of Means, Total 


135 


42.33 


86.41 


49.62 


49.89 






PRIVATE INSTITUTIONS 






5 


40+ 


41 


44 . 44 


95.34 


58.46 


61.93 


4 


20-39 


43 


43.19 


87.12 


50.81 


52.09 


3 


10-19 


63 


41.4rj 


83.81 


49.75 


49.49 


2 


5-9 ) 


38 


39.89 


80.71 


45.97 


45.55 


1 


1-4 j 












Mean of 


Means, Total 


185 


43.49 


86.74 


51.25 


52.26 






ALL INSTITUTIONS (Means 


of Means) 






5 


40+ 


116 


46.82 


91.06 


k.83 


57.95 


4 


20-39 


65 


42.96 


87.26 


50.70 


50.77 


3 


10-19 


87 


43.39 


86.22 


50.73 


51.28 


2 


5-9) 


52 


38.48 


81.78 


45.48 


44.31 


1 


1-4 J 












Mean of 


Means, Total 


320 


42.91 


86.58 


50.44 


51.08 





SIGNIFICANCE 


TESTS 


(F-ratios) 






Effect 


ndf ’s 
3, 312 


9,2** 


6.3** 


9.4** 


11.6** 


Stratum 


1, 312 


2.5 


1.3 


5.1 


4.1 


Type 

Stratum x Type 


3, 312 


4.5** 


4.8** 


4.1** 


3.0* 



^ Total number of FL majors 
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Table 5.4c 



Mean MLA Skills Test Scores, By Strattim 
and Institution Type 

SPANISH 



Stratum 


Size as 
of '62-'63l 


N 




MLA Skills 


Test Scores 




Listening 


Speaking 


Reading 


Writing 








PUBLIC INSTITUTIONS 






5 




40+ 


167 


44.86 


83.97 


47.92 


56.51 


4 




20-39 


72 


45.85 


88.76 


43.32 


56.94 


3 




10-19 


81 


42.79 


80.36 


46.75 


52.00 


2 




5-9 


70 


41.50 


81.76 


45.56 


50.61 


2. 




1-4 


25 


41.24 


81.16 


44.80 


48.28 


Meari 


of 


Means, Total 


415 


43.25 


83.20 


46.67 


52.87 










PRIVATE INSTITUTIONS 






5 




40+ 


78 


48.68 


88.87 


53.36 


59.36 


4 




20-39 


92 


46.24 


86.99 


49.52 


56.33 


3 




10-19 


101 


44.23 


83.42 


47.34 


53.95 


2 




5-9 


79 


42.02 


78.82 


44.80 


51.18 


1 




l~t> 


36 


44.11 


82.97 


46.11 


52.92 


Mean 


of 


Means, Total 


386 


45.06 


84.21 


48.23 


54.75 








ALL INSTITUTIONS (Means 


of Means) 






5 




40+ 


245 


46.77 


86.42 


50.64 


57.94 


4 




20-39 


164 


46.04 


87.88 


48 c 92 


56.64 


3 




10-19 


182 


43.51 


81.89 


47.04 


52.98 


2 




5-9 


149 


41.76 


80.29 


45.18 


50.89 


1 




1-4 


61 


42.67 


82.06 


45.45 


50.60 


Mean 


of 


Means , Total 


801 


44.15 


83.71 


47.45 


53.81 





SIGNIFICANCE 


, TESTS 


(F-ratios) 






Effect 


ndf ’s 










Stratum 


4, 791 


14.3** 


11.0** 


9.5** 


15.4** 


Type 


1, 791 


14.9** 


2.3 


10 . 1** 


5.1** 


Stratum x Type 


4, 791 


2.3 


3.5** 


3.9* 


1.2 



^Total number of FL majors 
**p < ,01 *p < .05 
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hlgher means on the MLA slcills tests. That is, it is conceivable that if all students 
at all participating institutions could have been tested ^ the addition of the weaker 
students at the larger institutions would reduce or even eliminate the institutional 
size differences we have just demonstrated. There is no sure way of determining to 
what extent this could have been the case. It has already been noted, however, that the 
amount of bias Introduced by student non-response is probably fairly small. It is not 
likely that differences in student response rate could account for all the differences 
among mean MLA skills test scores at institutions of different sizes. 

The data collected in this study make it possible to investigate other sources of 
variation among institutions of different sizes. From responses on the Questionnaire 
for Foreign Language Majors, a number of variables were created that reflect character- 
istics of the students and of the kind of instruction to which they had been exposed. 
Students in the various cross-classifications of institution size and type were compared 
on a number of these variables, with results as shown in Table 5.5. This table co/ntalns 
only the F-ratios computed from the two-way analyses of variance, and indications of 
the extent to which these F-ratios were statistically significant, making it possible 
to identify those variables that appear to differentiate institutions of different sizes 
otH types. (For comparative purposes, F-ratios are also given for the four MLA skills 
tests. These ratios are slightly different from those presented in Table 5.4 because 
of slight differences in the number of cases used for this new analysis.) 

Chapters VI and VII will discuss the student and instructional variables more 
fully, but for the time being it will suffice to give brief descriptions of them: 

Time Began : Coded, 1, 2, or 3 respectively depending upon whether the student start- 

ed the study of his major foreign language in "grade school", in high school, or in 
college. 

Time Abroad : Coded 1 for the student who had never been abroad to the country where 

the foreign language is used, 2 for the student who had toured in that country and/or 
had a summer course there, and 3 for the student who had had a year of study abroad. 

Use FL at Home : The question was "Do you and/or your parents speak your major lan- 
guage at home?" Answers were coded: 1 for "No", 2 for "Yes, occasionally," and 3 for 

"Yes, frequently." 

Hours College GrawwAr Couraea : Total number of course— hours taken in beginning, 

intermediate, or advanced courses devoted primarily to language study (as contrasted with 
literature). For convenience these are called "grammar courses" here. 

Teacher FL Use ( Grammar Courses).: Coded 1, 2, or 3 depending on how much the teacher 

was reported as using the foreign language in the classroom, averaged over "grammar 
courses as defined above. 

Student FL Use (Grammar Courses) : Coded 1 or 2 depending on how much the student 

reported he was required to speak the foreign language in the classroom, averaged over 
"grammar" courses . 

Tjingtiflge Laboratory Use (Grammar Courses): Coded 1, 2, or 3 depending upon the ex- 

tent to which use of a language laboratory was reported as being "an important and int.e- 
gral part" of the course, averaged over "grammar" courses. 

Teacher's Accent (Grammar Courses) : Coded 1 , 2 , or 3 depending upon the extent to^^ 
which the teacher was reported as being a native speaker of the FL, or having a native 
accent, averaged over "grammar" courses. 

Hours College Literature Courses : Total number of course-hours taken in literature 

courses at the college level. A literature course was defined in the questionnaire as 
"devoted primarily to the reading of texts in the foreign language (not in translation) , 
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Table 5.5 

F-Ratlos for Stratum and Institution-Type Effects 

for Selected Variables 



French German Spculsh 



Stratum Type S 


X T 


Stratum Type S 


X T 


Stratum Type S 


X T 


Variable 














VO 

o 


790 4, 


790 


ndf's 4, 1090 1, 


1090 4, 


1090 


3, 311 li 


311 3, 


311 


Time Began 


1.2 


11.5** 


1.5 


0.4 


2.0 


0.9 


2.5* 


2.1 


0.6 


Time Abroad 


6.7** 


46.4** 


2.7* 


1.7 


13,4** 


7.7* 


3.1* 


21.6** 


1.8 


Use FL at home 


1.8 


0.7 


1.0 


2.5 


2.0 


0.1 


2.5* 


3.9* 


1.9 


Hra. Coll. 
Gram. Courses 


8.1** 


4.1* 


2.3 


2.4 


1.5 


1.0 


1.7 


11.1** 


1.5 


Teacher FL Use 
(Gram. Courses) 


6.7** 


30.5** 


2.0 


2.4 


7.7** 


0.4 


5.5** 


6.2* 


1.2 


Student FL Use 
(Gram. Courses) 


4.9** 


34.2** 


1.2 


2.0 


4.3* 


0.8 


4.2** 


4.3* 


0.8 


Lang. Lab. Use 
(Gram. Courses) 


2.5* 


0.0 


5.7** 


5.3** 


2.1 


3.2* 


.3.5** 


11.4** 


2.4* 


Teacher's Accentb.O** 
(Gram. Courses) 


2.8 


1.4 


0.7 


1.7 


0.2 


2.7* 


3.5 


2.0 


Hrs. College 
Lit. Courses 


7.3** 


58.3** 


1.3 


1.7 


2.5 


4.6** 


4.3** 


21.0** 


3.4** 


Teacher FL Use 
(Lit. Courses) 


5.1** 


12.3** 


2.5* 


1.4 


2.4 


6.7** 


5.2** 


1.2 


5.2** 


Student FL Use 
(Lit. Courses) 


6.2** 


20.1** 


4.3** 


0.7 


2.4 


4.9** 


2.7* 


3.6 


4.0** 


Lang. Lab. Use 
(Lit. Courses) 


12.7** 


0.1 


7.0** 


4.8** 


0.4 


1.2 


15.0** 


6.4* 


5.5** 


Teacher's Accent2.9** 
(Lit. Courses) 


9.6** 


0.6 


1.5 


0.6 


0.7 


1.0 


0.8 


5.2** 


Hrs. Mlsc. 
Courses 


1.0 


1.2 


0.6 


0.5 


1.3 


2.3 


0.7 


9.0** 


7.5** 


MLA Listening 
Test 


29.1** 


59.0** 


1.2 


8.5** 


2.7 


4.6** 


14.5** 


15.4** 


2,4* 


MLA Speaking 
Test 


22.2** 


10.5** 


0.7 


5.9** 


1.4 


4.8** 


11.2** 


2.6 


3.7** 


MLA Reading 
Test 


21.8** 


58.9** 


0.8 


9.0** 


5.2* 


4.1** 


9.8** 


10.7** 


4.3** 


MLA Writing 
Test 


23.8** 


36.5** 


1.6 


11.1** 


4.3* 


3.0* 


15.5** 


5,3* 


1.3 


^.05 


2.38 


3.85 


2.38 


2.61 


3.87 


2.61 


2.39 


3.86 


2.39 


^.01 


3.34 


6.66 


3.34 


3.85 


6.72 


3.85 


3.35 


6.67 


3.35 
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rather than the study of the language as such." 

Teacher FL Use (Literature Courses) ; Same as Teacher FL Use (Gratamar Courses) ex- 
cept that it was averaged over literature courses. 

Student FL Use (Literature Courses) ; Analogous to Student FL Use (Grammar Courses) . 

Language Laboratory Use (Literature Courses) : Analogous to the same variable for 

Grammar Courses. 

Teacher *s Accent (Literature Courses) : Analogous to the same variable for Grammar 

Courses. 

Hours Miscellaneous Courses ; Hours in any foreign language courses not classified 
by the student as either beginning, intermediate, or advanced language courses, or as 
literature courses. 

It will be seen from Table 5.5 that every one of these variables yielded one or 
more significant F-ratlos for one or more design effects in one or more languages. 

Stratum differences were particularly pronounced in French, less so in Spanish, and only 
occasionally in Germano Institution type differences occurred frequently in both French 
and Spanish and also occasionally in German. There were scattered interaction effects 
of significance in all the languages. In order to analyze the nature of these effects, 
we present Table 5.6, which gives the means of selected variables within cross-^classlfl- 
catlons by stratum and institution type. These are the variables that appear to yield 
significant effects most consistently across languages, or that have other features of 
interest. Thus, the first two variables were Included (even though Time Began shows 
only one significant F-ratlo) because (as will be shown in Chapter Vll) , both had been 
found to be strongly associated with foreign language skill attainment. 

Technical Note ; The values labeled "Means of Means" in Table 5.6 are just 
that; they are not means obtained by pooling cases, because such pooling would 
obscure the contrasts that are tested by the two-way analysis of variance pro- 
cedure, which assumes that there are equal numbers of cases in each cross-classi- 
fication and thus that the cases available in each cross-classification are repre- 
sentative of the population that would fall in that cell. 

We will discuss the results in Table 5.6 by examining each variable in turn. 

Time Began . The overall means Indicate that there are more students who started 
early (in grade school or in high school) in French and Spanish than in German. Only 
in Spanish are there even barely significant differences among strata with respect to 
this variable; early starters show a slight tendency to go to larger institutions. In 
French there is a highly significant difference between public and private Institutions. 
The plain fact seems to be that students who start early in French are much more likely 
to go to private institutions . This probably reflect'!! the fact that FLES programs (at 
least those in the middle and late 1950* s, when these students were in elementary school) 
are more likely to be found in the wealthier schools or school districts ("well-heeled 
suburbs," as they are sometimes called) — districts that are more likely to send their 
college-bound students to private institutions. There is a slight trend of the same 
sort in Spanish, but it is not significant. (From data to be presented later, this may 
reflect a difference associated with geography. French is more likely to be studied 
in the Eastern part of the U.S., where there are more private institutions. Spanish 
FLES is more widely distributed throughout the U.S., and its products are more evenly 
distributed between private and public Institutions.) 

Since Time Began is quite strongly associated with foreign language attainment (see 
Chapter VII), we may infer, at this point, that one of the reasons why French students 
at private institutions are so superior in foreign language skills to those at public 
institutions (see Table 5.4) in that oh the average they started their foreign language 
study at an earlier stage. 
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Table 5.6a 

Means of Selected Students and Instructional Variables, 
By Stratum and Institution Type 



FRENCH 



Stratum 


St 

N 


;udent Variables 


Instructional Variables 




Time Time 

Began Abroad 


Teacher ' s 
Language 

(G) 


Lang. 

Lab. 

(G) 


Hrs. Teacher's 
Coll. Lang. 

(L) 


Lang. 

Lab. 










PUBLIC INSTITUTIONS 










5 (Large) 


203 


2.07 


1.60 


2.38 


1.68 


15.5 


2.57 


1.01 


4 


66 


2.12 


1.41 


2.41 


1.69 


17.4 


2.56 


1.11 


3 


61 


2.11 


1.44 


2.32 


1.70 


15.4 


2.48 


1.36 


2 


51 


2.10 


1.59 


2.48 


1.53 


15.6 


2.67 


1.12 


1 (Small) 


27 


2.04 


1.30 


2.36 


2.02 


12.1 


2.61 


1.12 


Mean of Means 


408 


2.09 


1.47 


2.39 


1.72 


15.2 


2.58 


1.14 








PRIVATE INSTITUTIONS 










5 (Large) 


111 


1.86 


1.92 


2.69 


1.54 


20.0 


2.67 


1.06 


4 


188 


2.01 


1.98 


2.68 


1.78 


20.1 


2.69 


1.17 


3 


198 


1.96 


1.89 


2.50 


1.69 


20.8 


2.78 


1.15 


2 


111 


2.05 


1.67 


2.64 


1.90 


18.0 


2.79 


1.26 


1 (Small) 


84 


2.13 


1.45 


2.29 


1.70 


17.5 




1.11 


Mean of Mearis 


692 


2.00 


1.78 


2.56 


1.72 


19.3 


2.68 


1.15 






ALL ; 


[NSTITUTIONS (Means of Means) 








5 (Large) 


314 


1.96 


1.76 


2.54 


1.61 


17.8 


2.62 


1.04 


4 


254 


2.06 


1.70 


2.54 


1.74 


18.7 


2.62 


1.14 


3 


259 


2.03 


1.66 


2.41 


1.70 


18.1 


2.63 


1.26 


2 


162 


2.07 


1.63 


2.56 


1.71 


16.8 


2.73 


1.19 


1 (Small) 


111 


2.08 


1.38 


2.32 


1.86 


14.8 


2.53 


1.12 


Mean of Means 


1100 


2.05 


1.62 


2.48 


1.72 


17.3 


2.63 


1.15 






SIGNIFICANCE 


TESTS (F-Ratios) 








Effects 


ndf's 
















S 


4, 1090 


1.2 


6.7** 


6.7** 


2.5* 


7.3** 


5.1** 


12.7** 


T 


1, 1090 


11.7** 46.4** 


30.5** 


0.0 


58.3** 


12.3** 


0.1 


S X T 


4. 1090 


1.5 


2.7* 


2.0 


5.7** 


1.3 


2.5 


7.0** 
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Table 5.6b 



GERMAN 



Student Variables Instructional Variables 



Time Time Teacher's Lang. Hrs. Teacher's Lang. 

Stratum N Began Abroad Language Lab. Coll. Lang. Lab. 

(G) (G) (L) (L) 



PUBLIC INSTITUTIONS 



5 (Large) 


75 


2.48 


1.48 


2.01 


1.43 


14.2 


2.29 


1.04 


4 


22 


2.68 


1.64 


2.17 


1.54 


16.9 


2.41 


1.01 


3 


24 


2.62 


2.21 


2.00 


1.93 


19.9 


2.66 


1.07 


' ] 
1 (Small) J 

Mean of Means 


14 


2.57 


1.57 


1.94 


1.38 


18.6 


2.11 


1.07 


135 


2.59 


1.72 


2.03 


1.57 


17.4 


2.37 


1.05 







PRIVATE INSTITUTIONS 










5 (Large) 


41 


2.51 


2.29 


2.31 


1.57 


19.9 


2.73 


1.04 


4 


43 


2.51 


1.91 


2.39 


1.65 


16.7 


2.53 


1.03 


3 


63 


2.41 


1.90 


2.10 


1.73 


18.1 


2.22 


1.04 


^ 1 
1 (Small) ) 

Mean of Means 


37 


2.45 


1.84 


2.09 


1.89 


16.9 


2.36 


1.24 


184 


2.47 


1.98 


2.22 


1.71 


17.9 


2.46 


1.09 







ALL INSTITUTIONS 


(Means of Means) 








5 (Large) 


116 


2.50 


1.88 


2.16 


1.50 


17.0 


2.51 


1.04 


4 


65 


2.60 


1.78 


2.28 


1.60 


16.8 


2.47 


1.02 


3 


87 


2.52 


2.06 


2.05 


1.83 


19.0 


2.44 


1.06 


2 } 


51 


2.51 


1.70 


2.02 


1.63 


17.8 


2.24 


1.16 


1 (Small) J 


















Mean of Means 


319 


2.53 


1.85 


2.13 


1.64 


17.7 


2.41 


1.07 






SIGNIFICANCE 


TESTS (F-Ratlos) 








Effects 


ndf's 
















S 


3, 311 


0.4 


1.7 


2.4* 


5.3** 


1.7 


1.4 


4.8** 


T 


1, 311 


2.0 


13.4** 


7.7** 


2.1 


2.5 


2.4 


0.4 


S X T 


3, 311 


0.9 


7.7** 


0.4 


3.2* 


4.6** 


6.7** 


1.2 
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Table 5.6c 



SPANISH 



Student Variables Instructional Variables 



Time Time Teacher's Lang. Hrs. Teacher's Lang. 

Stratum N Began Abroad Language Lab. Coll. Lang. Lab. 

(G) (G) (L) (L) 



PUBLIC INSTITUTIONS 



5 (Large) 


167 


2.04 


1.66 


2,50 


1.40 


15.50 


2.74 


1.02 


4 


72 


2.08 


1.64 


2.44 


1.40 


19.69 


2.81 


1.05 


3 


81 


2.17 


1.47 


2.29 


1.55 


16.97 


2.46 


1.02 


2 


70 


2.19 


1.57 


?,.24 


1.46 


14.71 


2.53 


1.12 


1 (Small) 


25 


2.08 


1.52 


2.02 


1.62 


14.17 


2.25 


1.20 


Mean of Means 


415 


2.11 


1.57 


2.30 


1.49 


16.21 


2.56 


1.08 







PRIVATE INSTITUTIONS 










5 (Large) 


77 


1.96 


2.08 


2.48 


1.46 


20.20 


2.63 


1.01 


4 


92 


2.00 


1.84 


2.54 


1.74 


19.15 


2.69 


1.04 


3 


101 


2.08 


1.71 


2.50 


1.62 


18.68 


2.61 


1.12 


2 


79 


2.14 


1.59 


2.36 


1.72 


19.34 


2.65 


1.35 


1 (Small) 


36 


2.22 


1.72 


2.29 


1.54 


15.67 


2.72 


1.10 


Mean of Means 


385 


2.08 


1.79 


2.43 


1.62 


18.61 


2.66 


1.12 







ALL INSTITUTIONS 


(Means of Means) 








5 (Large) 


244 


2.00 


1.87 


2.49 


1.43 


17.85 


2.68 


1.02 


4 


164 


2.04 


1.74 


2.49 


1.57 


19.42 


2.75 


1.04 


3 


182 


2.12 


1.59 


2.40 


1.58 


17.82 


2.53 


1.07 


2 


149 


2.16 


1.58 


2.30 


1.59 


17.02 


2.59 


1.23 


1 (Small) 


61 


2.15 


1.62 


2.16 


1.58 


14,92 


2.48 


1.15 


Mean of Means 


800 


2.10 


1.68 


2.37 


1.55 


17.41 


2.61 


1.10 






SIGNIFICANCE 


TESTS (F-Ratios) 








Effects 


ndf 's 
















S 


4, 790 


2.5* 


3.1* 


5.5** 


3.5** 


4.3** 


5.2** 


15.0** 


T 


1, 790 


2.1 


21.6** 


6.2* 


11.4** 


21.0** 


1.2 


6.4* 


S X T 


4. 790 


0.6 


1.8 


1.2 


2.4* 


3.4** 


5.2** 


5.5** 
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Tlme Abroad . This variable yielded highly significant differences in institution 
types in all three languages; students at private institutions have on the average more 
experience in travel or study abroad. Since this variable is also known to be very 
strongly associated with foreign language skill attainment (see Chapter VII) • it consti- 
tutes another basis for inferring that students at private institutions show better 
foreign language skill attainment because they are more likely to have studied or trav- 
elled abroad. 

In French and Spanish, there is a significant stratum difference whereby students 
at the larger Institutions, particularly the private ones, are more likely to have had 
experience abroad. In German, no overall stratum effect exists, but the interaction ef- 
fect indicates that in private institutions the travelled students tend to be at the 
larger institutions while at public institutions they tend to concentrate in medium-sized 
ones. Again, these differences are likely to be associated with economic factors; it 
costs more money to go abroad to travel or study. The stratum differences in French and 
Spanish possibly reflect a greater likelihood that the larger institutions have programs 
for study abroai^. 

Although in Table 5.6 Time Began and Time Abroad are labeled as "student variables," 
this is only because they pertain to possible experiences of the student before he 
reaches the college level. Actually, they are Instructional variables, of a sort, and 
the Time Abroad variable is most likely to reflect experience during the college years. 

We turn now to instructional variables that clearly pertain solely to experiences 
during the college years. 

Teacher FL Use (GrAnmar Courses) . This variable yields significant institution type 
differences in all languages. Teachers in private institutions are more likely than those 
in public institutions to use the foreign language in the classroom, at least for the 
courses actually taken by the students in the sample. (The fact that the mean for this 
variable is particularly high in French may reflect the fact that the French students are 
more likely to be advanced students by the time they arrive at college.) There are also 
stratum differences in all languages, but in French these differences are apparently not 
perfectly correlated with the size of the institution. There tends to be most use of 
French in the classroom in stratum 2, the next to smallest; after this come strata 4 and 
5, the two largest strata, then stratum 3, and finally stratum 1, the smallest (see the 
means of means given for "all institutions"). On the other hand, in Spanish (and also in 
German) there is a trend whereby the foreign language is more often used in the classroom 
in the larger institutions than in the smaller institutions. Insofar as the teacher's 
use of the foreign language in the classroom may be a factor in the student's attainment 
of foreign language skill, this is another possible cause of differences in the mean 
skill attainment shown by students at different types and sizes of institutions. 

Language Laboratory Use (Grammar Courses) . The relationships exhibited by this 
variable are very complex, and seem to be different across languages. In French, there 
is no overall difference between private and public institutions, but there are signifi- 
cant stratum and stratum x type effects whereby the smaller institutions, particularly 
the i?ublic ones, are more likely to use language laboratories as an Integral part of a 
language course. Possibly in this way they attempt to substitute for the lack of teachers 
with a native accent. A somewhat similar phenomenon occurs in German, except that strik- 
ingly high use of the language laboratory occurs in public institutions of medium size 
(stratum 3) and in private institutions of small size (strata 1 and 2) . In Spanish the 
major effect is associated with institution type, although there are also stratum and 
interaction effects: private institutions, particularly the smaller ones, tend to use 

language laboratories more often. 

Hours of College Literature Courses . Here again the differences are of a complex 
order and vary from language to language. In French and Spanish both stratum and insti- 
tution type differences are found: students at private institutions nearly always report 

more hours in literature courses, and the average number of hours increases with the size 
of the institution. In German, on the other hand, the only effect is an interaction one 



1 
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whereby more hours are reported in smaller public institutions and larser private insti- 
tutions. 

Teacher FL Use (Literature Courses) . In French, private institutions show higher 
overall averages in this variable, but the trends by strata are reversed { that is, in 
private institutions the averages are positively correlated with institution size but in 
public institutions they are negatively correlated with institution size. A similar 
interaction effect occurs in German and Spanish. In Spanish there is a significant 
stratum effect (overall, larger institutions are higher on this variable), but no overall 
institution type difference. 

Language Laboratory Use (Literature Courses) . The overall magnitude of the means 
(in the range near 1) indicates that there is seldom any use of the language laboratory 
in literature courses. Where it occurs, it is most likely to occur in the smaller or 
medium-small institutions, and this effect is highly significant in all languages. 

In view of these findings, it is not surprising that institutions of different sizes 
and types vary considerably in the mean attainment of these students in foreign language 
Our best guess is that even if there had been a uniformly high student response 
rate in all institutions, the various strata and institution types would still have shown 
highly significant differences in foreign language test scores. 

As one test of this possibility, a special analysis of institutional differences in 
FL skill attainment was made with a statistical control for two student variables that 
are strongly associated with FL skill attainment: Time Began and Time Abroad. That is, 

the cross-classification means for the MLA skills test ucores were adjusted for any ef- 
fects that might have been associated with differences in the time students began study 
of their major foreign language or in the amount of travelling and studying abroad they 
had done. The results are shown in Table 5.7. The adjustments tended to reduce the size 
of the F-ratlos, particularly those associated with interaction effects. As expected, 
however, both stratum and institution-type differences tended to remain significant, with 
the same trends as observed before. The effect of stratum was most resistant to the ef- 
fects of the adjustment; in fact, in German the stratum difference was enhanced by the 
adjustment. Institution-type differences, were virtually eliminated in the German and 
Spanish groups but remained significant (although not nearly as strongly) in Frendi, ex- 
cept in the case of the Speaking test, where the difference disappeared after the ad- 
justment. The conclusion suggested by these findings is that larger institutions, and 
to some extent the private ones, prod^ice better language students than smaller ones, even 
after student backgrounds are taken into account. 

3. Variations in Foreign Languaee Skill Associated with Geographical Region 

Geographical region of the U. S. was not explicitly used as a basis for the sampling 
design, since institutions were sampled at random from all parts of the country. It is 
of Interest, nevertheless, to investigate any variations in foreign language skill at- 
tainments that mlg>t be associated with differences in geographical region of the insti- 
tutions sampled. For this purpose, all "regular" cases in French, German, and Spanish 
with complete information on MLA skills test scores were classified as to which of eight 
geographical regions their institutions were located in. These geographical regions are 
those used in certain tabulations of the U. S. Office of Education, as follows: 

1. New England : Conn., Maine, Mass., N. H., R. I., Vt. 

2. Mideast : Del., D. C. , Md., N. J., N. Y., Penna. 

3. Great Lakes : 111., Ind., Mich., Ohio, Wise. 

4. Plains : Iowa, Kans., Minn., Missouri, Neb., N. D., S. D. 

5 . Southeast : Ala . , Ark . , Fla . , Ga. , Ry • f La . , Miss ., N. C., S. C., Tenn . , Va. , 

W. Va. 





F-Ratios for Stratum and Institution-Type Effects 
for MLA Skills Test Scores, Without and With Adjustment for Variations 
in the Time the Student Began FL Study and in Amount of Travel and Study Abroad 
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6. Southwest : Ariz., N. M. , Okie., Tex. 

7. Rocky Mountain : Colo.» Idaho» Mont., Utah, Wyo. 

8. Far West and Pacific : Alaska, Calif., Hawaii, Nev., Ore., Wash. 

Table 5.8 shows the percentages of French, German, and Spanish majors 
of these eight regions, and the mean scores of these majors on each of the MLA sKiiis 

tests in the relevant language. 



The percentages reflect in part, of course, the geographical distribution of insti- 
tutions over these regions. For example, there are large numbers of institutions in the 
Mideast region, and many of these institutions were included in the national study sam- 
ple. There are interesting variations, however, in the distribution of FL majors over 
the regions. Examining only percentages (that is, laying aside the fact that French is 
overall the most popular language, Spanish the next most popular, and German a trailing 
third) we find that French is relatively more popular than Spanish or German in the New 
Eneland and Mideast states. Spanish is relatively more popular in the Plains, Southeast, 
and Southwest states, and German relatively more popular in the Great Lakes, Rocky Moun- 
tain and Far West and Pacific states. These differences correspond, roughly, to the 
known orientations of the different regions towards the different languages. The North- 
east has traditionally had closer ties with France and (tc extent) French Canada^ 

the Southeast and Southwest have strong Spanish Influences; in their histories and con-, 
temporary lives; the Great Lakes states were those where large ©umbers of German immi- 
grants went to settle. 

Regional differences in MLA skills test scores are in every case statistically sig- 
nificant, highly so in French and Spanish where results are available for large numbers 
of students. Generally, the means are higher in those regions where the languages are 
relatively more popular. At the same time, regions appear to differ in their overall 
performance, regardless of language. For example, students in New Engird institutions 
are generally high in all languages; only in Spanish are they slightly less superior than 
students in some other regions. Students in the Plains, Southeast, and Southwest states 
are generally poorer than students from other regions, except that students of Spanish 
in the Southwest do quite well. 



It should be remembered that these regional differences are probably associated in 
a complex way with other variables mentioned in this chapter: institution size, insti- 

tution type, the student’s time of beginning the foreign language, the student s experi- 
ences in study or travel abroad, etc. No attempt has been made in the present report to 
study regional differences adjusted for the effects of these other variables. 



4. Estimated National Norms for MLA Skills Tests 

Chapter IV, Section 3 presented means, standard deviations, and percentile norms de- 
rived directly from data obtained from the students actually tested. Because the strati- 
fied sampling design represented institutions of different sizes at different rates, and 
because student response rate Itself varied over the cells of the sampling design, it was 
thought desirable to attempt to estimate distributions of test scores for the populations 
which had been sampled. This would result in national norms that should be good esti- 
mates of the norms that would have been obtained had all students in the defined popula- 
tion been tested in the spring of 1965. 

Consultations were held with Professor W. G. Cochran of the Harvard University De- 
partmexit of Statistics and with Dr. Theodore Colton of the Harvard Medical School as to 
the optimal estimation procedures to be used. After considerable study of the problem, 
it was decided not to make any attempt to correct for the effects of student non-response 
upon test score distributions, since Information as to the amount of such effects was 
sparse, and since the meager information that did exist (see Section 1 of this chapter) 
indicated that the effects might be relatively small. 
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Table 5.8 

Mean MLA Skills Test Scores by Geographical Region 

(Regular cases) 





M 


% 


Listening 


Speaking 


Reading 


Writing 






FREMCH 








1. Mew England 


155 


13.9 


46.32 


74.86 


51.57 


50.12 


2. Mid East 


335 


30.0 


44.67 


74.55 


49.41 


48.11 


3. Great Lakes 


255 


22.9 


. 44.19 


73.58 


47.90 


46.57 


4. Plains 


83 


7.4 


41.76 


71.83 


47.13 


43.71 


5. Southeast 


148 


13.3 


41.35 


70.40 


46.78 


44.60 


6. Southwest 


25 


2.2 


40.64 


71.16 


46.40 


43.52 


7. Rocky Mountain 


34 


3.0 


46 . 00 


75.74 


49.29 


47.56 


8. Far West & Pacific 


80 


7.2 


45.14 


74.25 


49.80 


48.55 


Total 


1115 


100.0 


44.12 


73.56 


48.81 


47.16 


P 






< .001 


< .001 


< .001 


< .001 






GERMAM 








1. Mew England 


36 


11.1 


45.44 


88.39 


54.11 


54.67 


2. Mid East 


64 


19.8 


43.05 


86.70 


50.22 


51.64 


3. Great Lakes 


82 


25.3 


44.34 


87.80 


51.76 


53.30 


4. Plains 


33 


10.2 


39.76 


82.03 


48.24 


47.42 


5. Southeast 


31 


9.6 


40.13 


84.39 


48.61 


47.94 


6 . Southwest 


7 


2.2 


41.71 


84.29 


46.43 


48.71 


7. Rocky Mountain 


29 


9.0 


45.48 


91.28 


52.76 


56.71 


8. Far West & Pacific 


42 


13.0 


, 44.76 


86.10 


51.74 


52.64 


Total 


324 


100.0 


43.44 


86.75 


51.03 


52.09 


P 






< .05 


> .05 


> .05 


> .05 






SPANISH 








1. Mew England 


61 


7.5 


46.39 


85.67 


51.00 


57.54 


2. Mid East 


219 


26.9 


44.09 


80.65 


47.37 


54.77 


3. Great Lakes 


134 


16.4 


43.60 


83.80 


46.25 


53.77 


4. Plains 


102 


12.5 


41.81 


80.75 


44.19 


51.16 


5. Southeast 


109 


13.4 


42.98 


83.64 


46.65 


51.94 


6 . Southwest 


70 


8.6 


45.79 


88.87 


49.86 


57.34 


7 . Rocky Mountain 


45 


5.5 


47.73 


92.04 


50.89 


57.80 


8. Far West & Pacific 


75 


9.2 


48.11 


88.25 


52.05 


57.55 


Total 


815 


100.0 


44.47 


83.99 


47.80 


54.60 


P 






< .001 


< .001 


< .001 


< .001 



o 




m 
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The estimctlon procedure finally decided upon consisted of two adjustments for ef- 
fectl« sXSng Iractlons: (1) the'ratlo of the number (B) of 

in a particular language, stratum, and institution type, to Ssted in 

ents that according to U. S. Office of Education figures were availa b ja to be tested in 

that language, stratum, and institution type in the participating 
Mtlo, C/B, made a correction for student response rate within a given 

and institution type classification; the reciprocal of «•« "f cOT^ct- 

correction for the effective sampling fraction In the sampling design, ie e, 

ed for the fact that the sampling design represented smaller-sized Institutions much less 

than larger-sized Institutions, 

The ratios thus determined are shown In Table 5.9. The reciprocals of the ratios 
were applied to the score frequencies for a given language a"d test 
frequency for the national population In that language, according to the formula 

r C \ ^ E V 

^In “ ^Is ^ B D ^ * 

where is the frequency of scores in the ith score interval in the estimated national 
frequency distribution In a given language and test, and fig Is the frequency o 
in the i**h score interval in the sample actually tested in a g ven angu ge ,,, 

The ratio C/B was of course different for each test, language, stratum, and InstitutiOT 
type; the ratio E/D was varied only by stratum and institution type; i.e., it was raiform 
f« ;il languages, on the assumption that the data availAls in a given stratum and in- 
stitution classification should be boosted equally for all languages. 



It should be noted that the nari.er B. and the values fis were all “ 



•I 



reg- 

ular “asfs""^ ref"rdVr“iousIy:'^T^;t"ls:“L^^^ speakers and other -odd" 

types of cases with unusual language experiences had been national 

norms are therefore for a population composed exclusively of regular 

An example may help In understanding this procedure. There was 1 student In 
Strati Twho obtSned I score of 54 on the French Listening test, and there were 2 stud- 
en^f who obtained this score In Private Stratum 1. Now, In Public Stratum 1 there were 
In all 29 students tested (B) out of a total of 31 (C) available to be tested In the par- 
ticipating Institutions of that stratum. There were 11 (D) 

total of 75 (E) public Institutions In that stratum. The ratio (B/C) was .9355, and the 
T^lo (D/E) was !i 467, as may be found In Table 5,9. By the formula given f ove we may 
tten estimate the total number of students In Public Stratum 1 who would get a score of 
5*i as l/[(. 9355) (.1467)1 ■ 7.2884. Similarly, In Private Stratum 1 there were 
students tested (B) out of a total of 97 (C) available to be tested In the 
Institutions of that stratum. There were 32 (D) such 

(E) private Institutions in that stratum. The ratio (B/C) was .8969, and the ratio (D/E) 
was l27s! W^then estimate the total number of students In Private Stratum 1 who would 
get a score of 54 as 2/ [(.8969) (.1275) ] - 17.4908. Adding the estimates for public md 
Private Institutions, we have 7.2884 + 17.4908 - 24.7792 as the estimated number of stud- 
ents In stratum 1 of the national population who would obtain a score of 54 In the Fre 

Listening test. 

Summing these and similar computations for the remaining strata md for 
scores vielLd the estimated frequency distribution for the French Listening test in t^ 
national ooDulatlon. The estimated number of French students in the national populatlo 
Ts rtf .rof fr^qienS:., 5564.63. The mean and standard deviation 
score distribution were computed as 43.0766 and 7.8700, respectively, ( e 
number^of students In the three languages are roughly comparable to the actual numbers In 

the class of 1965; see Chapter III.) 
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Table 5.9 

Ratios Used In Estimation of National Norms 



A. Ratios B/C (see text), by language, test, stratum, and institution type 





PUBLIC INSTITUTIONS 




PRIVATE INSTITUTIONS 




Stratum 


L 


TEST 

S 


R 


W 


L 


TEST 

S R 


W 




5 (large) 


.294 


.271 


.296 


FRENCH 

.296 


.367 


.358 


.361 


.364 




4 


.449 


.376 


.449 


.449 


.523 


.512 


.542 


.539 




3 


.477 


.462 


.500 


.500 


.707 


.677 


.718 


.718 




2 


.724 


.684 


.711 


o711 


.818 


.790 


.8li 


.811 




1 (small) 


.935 


.903 


.935 


.903 


.897 


.866 


.897 


.897 




5 (large) 


.236 


.227 


.233 


GERMAN 

.230 


.459 


.429 


.449 


.459 




4 


.250 


.229 


.250 


.250 


.512 


.512 


.536 


.536 




3 


.429 


.429 


.429 


.429 


.759 


.736 


.747 


.759 




2 


.333 


.333 


.333 


.333 


.543 


.543 


.543 


.543 




1 (small) 


.500 


.500 


.500 


.500 


.833 


.792 


.833 


.833 




5 (large) 


.206 


.206 


.206 


RUSSIAN 

.206 


.271 


.271 


.271 


.257 




4 


.300 


.300 


.300 


.250 


.429 


.405 


.429 


.429 




3 


.500 


.500 


.500 


.500 


.250 


.250 


.250 


.250 




2 










2.000 


2.000 


2.000 


2.000 




1 (small) 


1.000 


1.000 


1.000 


1.000 












5 (large) 


.252 


.246 


.252 


SPANISH 

.257 


.410 


.405 


.405 


.410 




4 


.442 


.354 


.437 


.437 


.466 


.442 


.486 


.486 




3 


.500 


.494 


.512 


.512 


.582 


.560 


.571 


.577 




2 


.809 


.787 


.809 


.798 


.934 


.912 


.934 


.934 




1 (small) 


.619 


.595 


.619 


.619 


.720 


.720 


.720 


.720 





B. Ratios D/E, by stratum and institution type 



Stratum 


PUBLIC INSTITUTIONS 


PRIVATE INSTITUTIONS 


5 (large) 


.8000 


.9091 


4 


.4815 


.6047 


3 


.3778 


.3441 


2 


.1803 


.2239 


1 (small) 


.1467 


.1275 
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Table 5«10 shows the N'Sf means, and standard deviations estimated for all the MLA 
skills tests. Table 5.11 gives the percentile norms for each test. Each percentile rank 
is computed for the mid-point of the corresponding score interval. Table 12 gives the 
estimated percentile norms in a more convenient form than Table 11; they are analogous to 
the norms presented by Educational Testing Service (1964b) for NDEA Institute teachers 
(see Appendix E-3). Figures 5.1, 5.2, 5.3, and 5.4 give the norms in graphic form, plot- 
ted on scales that are set up to correspond with the FSI "absolute language proficiency 
scales; these charts also show the ranges for the Myers-Melton (1964) teacher qualifica- 
tion standards (Superior, Good, Minimal, and Unsatisfactory). For a discussion of the 
FSI and Ityers-Melton equivalences, see Chapter II, Section 2, of this report. 

The arrows depicted close to the names of the four skills at the top of each figure 
will remind the reader that the Listening and Speaking scales were equated to the FSI "S" 
(Speaking) ratings, while the Reading and Writing scales were equated to the FSI R 
(Reading) ratings. The scale for each MLA skill test is labeled with the so-called con- 
verted" scores, and superimposed on the scale are bars and other indications showing the 
median, the middle 50 per cent of the cases (from the 25th to the 75th percentile) and 
the middle 80 per cent of the cases (l.e., from the 10th to the 90th percentile). 

The most striking thing about these charts is the generally low median levels of at- 
tainment in audio-lingual skills that they reveal. The median graduate with a foreign 
language major can speak and comprehend the language only at about an FSI Speaking rating 
of "2+," that is, somewhere between a "limited working proficiency" and a "minimum pro- 
fessional proficiency.** There is some variation around this value for the different 
languages, which the reader may perceive by inspecting the charts. In Reading and Writ- 
ing, the French, German, and Spanish groups attain median scores that correspond approxi- 
mately to an FSI rating of **3,** that is, **minimum professional proficiency.** The slight- 
ly poorer showing of the Russian group in all four skills may reflect either the greater 
difficulty of the Russian language for English speakers or the relatively smaller invest- 
ment that has been made in the teaching of Russian in the United States, or both. But 
when we consider all the results, the net showing of college foreign language majors is 
not impressive. One would think that the median college foreign language major ought to 
attain more than a **mlnlmum professional proficiency** in basic language skills as a result 
of his efforts. We are speaking, of course, of medians; the distributions range from 
abysmally poor performance to quite superior performance — up to the limits measured by 
the tests, in some cases. 

Comparison of CELT Norms with NDEA Institute Norms 

For some years— that is, since about 1963, interpretations of scores on the MLA Ad- 
vanced Proficiency Tests have necessarily been based on the only norms available— the 
norms from teachers enrolled at NDEA Institutes in 1961, 1962, and 1963. (These norms 
may be found in Appendix E-3.) From the point of view of teacher certification and for 
other purposes, the College Foreign Language Testing (CFLT) norms presented in Tables 11 
and 12 will probably be more useful and pert#:ient than the NDEA norms. It is well, there- 
fore, to recognize the fairly considerable differences between the CFLT and the NDEA 
norms. Table 5.13 presents a statistical summary of the differences in terms of means 
and standard deviations. The CFLT norms are, of course, based on smaller actual numbers 
of cases tested, but for practical purposes they may be considered good estimates of the 
populations that were sampled, these populations being (by chance) approximately of the 
same magnitude, numerically, as the samples of teachers at NDEA Institutes in 1961—63. 

As a general statement to summarize the differences between the CFLT and the NDEA 
norms, it may be said that the CFLT samples show clearly higher means than the NDEA pre- 
test norms, and slightly higher means, in general, than even the NDEA post- test norms; 
further, they show much less range or variance than the NDEA norms. Despite what would 
appear to be considerable range of ability in the CFLT population, it is still smaller in 
every comparison than that shown by the NDEA norms. The CFLT group, it must be observed, 
could be said to have a relatively homogeneous background in the sense that every member 
has been recently through the experience of preparing himself as a candidate for an A.B. 
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Figure 5.1 



Estimated National Norms for CFLT Population, 
with MLA-FSI and Myers-Melton Equivalents 
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Figure 5.2 

Estimated National Norms for CELT Population, 
with MLA-FSI and Myers-Melton Equivalents 
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Figure 5.3 



Estimated National Norms for CFLT Population, 
with MLA-FSI and Myers-Melton Equivalents 
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Figure 5.4 
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Estimated National Norms for CFLT Population, 
with MLA-FSI and Myers-Melton Equivalents 
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Table 5.10 



Means and Standard Deviations for Estimated National Norms 
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Table 5.11 



Estimated National Noras for MLA Proficiency Test Skill Scores, 
College Senior PL Majors (Percentile Ranks of Converted Scores) 



Listening 



FRENCH 



Speaking 



Reading 



Score 


%ile 


Score 




Rank 




56 


99.3 


104 


55 


97.2 


103 


54 


93.8 


102 


53 


89.5 


101 


52 


85.5 


100 


51 


80.7 


99 


50 


75.6 


98 


49 


71.2 


97 


48 


66.9 


96 


47 


62.6 


95 


46 


58.4 


94 


45 


54.5 


93 


44 


51.1 


92 


43 


46.9 


91 


42 


42.6 


90 


41 


39.1 


89 


40 


35.7 


88 


39 


32.1 


87 


38 


29.0 


86 


37 


25.9 


85 


36 


22.3 


84 


35 


19.0 


83 


34 


16.3 


82 


33 


13.9 


81 


32 


11.3 


80 


31 


8.3 


79 


30 


5.7 


78 


29 


3.9 


77 


28 


2.5 


76 


27 


1.2 


75 


26 


0.4 


74 


25 


0.2 


73 


24 


0.1 


72 



71 

70 

69 

63 
67 
66 
65 

64 
63 
62 
61 
60 
59 
58 
57 
56 
55 
54 
53 
52 
51 
50 
49 
48 
47 
46 
45 
44 
43 
42 
41 
40 
39 



3.0 
2.5 
1.9 
1.4 

1.1 
0.9 
0.7 
0.6 
0.5 
0.5 
0.4 
0.3 
0.2 
0.2 
0.2 
0.2 
0.1 



Writing 



%ile 


Score 


%ile 


Score 


%ile 


Rank 




Rank 




Rank 


99.9 


69 


100.0 


70 


99.8 


99.8 


68 


99.9 


69 


99.4 


99.7 


67 


99.8 


68 


99.0 


99.6 


66 


99.4 


67 


98.8 


99.6 


65 


98.8 


66 


98.5 


99.5 


64 


98.1 


65 


98.1 


99.4 


63 


96.7 


64 


97.4 


99.2 


62 


95.1 


63 


96.4 


99.0 


61 


93.4 


62 


95.0 


98.8 


60 


91.1 


61 


93.4 


98.7 


59 


88.3 


60 


91.5 


98.3 


58 


85.4 


59 


89.5 


97.9 


57 


82.6 


58 


87.4 


97.3 


56 


79.3 


57 


85.2 


96.5 


55 


76.2 


56 


83.0 


95.8 


54 


73.0 


55 


80.4 


94.8 


53 


69.3 


54 


77.8 


93.4 


52 


65.2 


53 


74.9 


92.1 


51 


61.1 


52 


71.5 


90.7 


50 


57.2 


51 


67.8 


89.0 


49 


53.7 


50 


63.7 


87.1 


48 


49.9 


49 


60.1 


84.6 


47 


45.6 


48 


57.1 


82.0 


46 


41.7 


47 


54.0 


79.1 


45 


38.0 


46 


50.3 


75.5 


44 


34.4 


45 


46.7 


71.6 


43 


31.0 


44 


43.3 


67.2 


42 


27.6 


43 


39.3 


62.6 


41 


24.5 


42 


35.5 


58.5 


40 


21.2 


41 


32.0 


54.3 


39 


18.3 


40 


28.2 


49.9 


38 


15.8 


39 


24.9 


45.9 


37 


13.2 


38 


21.7 


42.1 


36 


10.9 


37 


18.7 


38.0 


35 


8.7 


36 


16.1 


34.3 


34 


6.4 


35 


14.0 


31.1 


33 


4.5 


34 


12.5 


27.5 


32 


3.1 


33 


10.5 


23.8 


31 


2.0 


32 


8.3 


20.2 


30 


1.5 


31 


6.5 


17.5 


29 


1.1 


30 


5.0 


15.2 


28 


0.6 


29 


3.9 


12.6 


27 


0.1 


28 


3.1 


10.3 






27 


2.2 


8.8 






26 


1.4 


7.4 






25 


0.7 


6.0 






24 


0.4 


4.9 






23 


0.2 


3.7 






22 


0.1 



IBKiilMIliitl 
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Table 5 . 11, continued 



Estimated National Norms for MLA Proficiency Test Skill Scores, 
College Senior FL Majors (Percentile Ranks of Converted Scores) 



FRiC 




Listening 



GERMAN 



Speaking 



Reading 



60 

59 

58 

57 

56 

55 

54 

53 

52 

51 

50 

49 

48 

47 

46 

45 



2.3 

1.4 
1.3 
1.2 

1.1 

1.1 

1.1 

1.1 

1.1 

1.1 

1.1 

1.1 
1.0 
0.9 
0.9 
0.4 



Writing 



Score 


%lle 


Score 


%lle 


Score 


%lle 


Score 


%lle 




Rank 




Rank 




Rank 




Rank 


56 


98.8 


118 


99.9 


70 


99.9 


77 


99.9 


55 


96.0 


117 


99.9 


69 


99.7 


76 


99.6 


54 


92.3 


116 


99.9 


68 


99.1 


75 


99.3 


53 


88.6 


115 


99.8 


67 


97.3 


74 


98.7 


52 


83.1 


114 


99.7 


66 


95.5 


73 


97.7 


51 


77.3 


113 


99.3 


65 


94.0 


72 


96.6 


50 


73.3 


112 


98.8 


64 


92.5 


71 


95.5 


49 


69.0 


111 


98.4 


63 


91.6 


70 


94.1 


. 4b 


64.7 


110 


97.6 


62 


90.0 


69 


92.5 


47 


61.7 


109 


96.7 


61 


87.8 


68 


90.7 


46 


58.5 


108 


96.1 


60 


85.3 


67 


88.8 


45 


53.6 


107 


95.4 


59 


80.6 


66 


87.1 


44 


49.7 


106 


94.6 


58 


76.4 


65 


85.4 


43 


45.8 


105 


94.1 


57 


73.7 


64 


82.3 


42 


43.0 


104 


93.3 


56 


69.7 


63 


79.2 


41 


41.5 


103 


91.4 


55 


65.2 


62 


76.5 


40 


39.8 


102 


88.2 


54 


61.3 


61 


73.0 


39 


37.3 


101 


85.6 


53 


58.5 


60 


69.7 


38 


34.7 


100 


84.9 


52 


55.6 


59 


66.8 


37 


31.8 


99 


84.4 


51 


52.1 


58. 


64.1 


36 


29.2 


98 


83.0 


50 


48.3 


57 


61.3 


35 


28.0 


97 


81.6 


49 


44.3 


56 


58.2 


34 


24.5 


96 


79.6 


48 


40.4 


55 


56.6 


33 


19.4 


95 


76.2 


47 


37.6 


54 


55.1 


32 


15.9 


94 


73.4 


46 


36.0 


53 


53.1 


31 


13.1 


93 


70.5 


45 


33.6 


52 


51.2 


30 


10.5 


92 


67.3 


44 


30.0 


51 


49.8 


29 


7.7 


91 


65.1 


43 


26.3 


50 


48.2 


28 


4.1 


90 


62.7 


42 


22.9 


49 


46.3 


27 


1.3 


89 


59.3 


41 


20.0 


48 


44.5 


26 


0.6 


88 


55.3 


40 


18.1 


47 


42.5 


25 


0.5 


87 


51.9 


39 


15.6 


46 


40.1 


24 


0.2 


86 


48.8 


38 


13.5 


45 


37.2 






85 


45.5 


37 


12.2 


44 


34.4 






84 


42.2 


36 


10.9 


43 


31.5 






83 


39.5 


35 


8.6 


42 


28.6 






82 


37.4 


34 


6.8 


41 


27.0 






81 


34.9 


33 


5.4 


40 


25.8 






80 


31.9 


32 


3,3 


39 


24.2 






79 


28.3 


31 


2.0 


38 


21,8 






78 


25.2 


30 


1.2 


57 


19.7 






77 


23.8 


29 


0.7 


36 


17.6 






76 


22.8 


28 


0.5 


35 


15.2 






75 


21.2 


27 


0.5 


34 


13.9 






74 


18.8 


26 


0.5 


33 


12.5 






73 


17.0 


25 


0.3 


32 


11.0 






72 


15.8 






31 


9.4 






71 


14.1 






30 


7.8 






70 


12.6 






29 


6.5 






69 


11.4 






28 


5.3 






68 


10.3 






27 


4.9 






67 


7.8 






26 


4.2 






66 


5.5 






25 


2.7 






65 


4.6 






24 


1.3 






64 


4.1 






23 


0.6 






63 


3.6 






22 


0.5 






62 


3.3 






21 


0.5 






61 


3.1 






20 


0.2 



irikMlIlltiSiiiM 






-91- 

Table 5.11, continued 

Estimated National Norms for MLA Proficiency Test Skill Scores, 
College Senior FL Majors (Percentile Ranks of Converted Scores) 



RUSSIAN 



Listening 



Speaking 



Reading 



Writing 



Score %ile 
Rank 

55 97.7 
54 95.5 
53 95.5 
52 94.4 
51 90.9 
50 87.3 
49 84.2 
48 80.0 
47 75.6 
46 71.7 
45 66.4 
44 58.1 
43 51.5 
42 44.3 
41 38.9 
40 35.6 
39 31.2 
38 28.4 
37 24.1 
36 19.2 
35 15.0 
34 7.9 
33 2.3 
32 1.2 
31 0.4 



Score 


%ile 

Rank 


Score 


115 


98.8 


65 


114 


97.6 


64 


113 


97.6 


63 


112 


97.6 


62 


111 


97.6 


61 


110 


97.6 


60 


109 


97.6 


59 


108 


97.6 


58 


107 


96.3 


57 


106 


95.1 


56 


105 


95.1 


55 


104 


94.4 


54 


103 


93.8 


53 


102 


93.8 


52 


101 


93.2 


51 


100 


91.5 


50 


99 


90.4 


49 


98 


90.4 


48 


97 


89.7 


47 


96 


87.8 


46 


95 


86.1 


45 


94 


85.3 


44 


93 


84.9 


43 


92 


84.9 


42 


91 


84.9 


41 


90 


84.1 


40 


89 


82.8 


39 


88 


80.4 


38 


87 


78.0 


37 


86 


74.7 


36 


85 


71.3 


35 


84 


69.2 


34 


83 


67.1 


33 


82 


65.8 


. 32 


81 


60.3 


31 


80 


54.6 


30 


79 


52.3 


29 


78 


47.0 


28 


77 


40.9 


27 


76 


37.1 


26 


75 


32.9 


25 


74 


29.5 


24 


73 


26.6 




72 


24.0 




71 


23.6 




70 


22.0 




69 


19.7 




68 


18.9 




67 


18.9 




66 


17.4 




65 


14.6 




64 


12.4 




63 


11.6 




62 


11.1 




61 


10.7 




60 


9.7 




59 


7.9 




58 


6.7 




57 


5.9 




56 


4.8 




55 


4.0 




54 


3.7 




53 


2.5 




52 


1.3 





%ile 


Score 


%ile 


Rank 




Rank 


99.4 


80 


98.8 


98.7 


79 


97.6 


98.7 


78 


96.5 


98.7 


77 


93.7 


98.3 


76 


91.6 


97.5 


75 


90.1 


95.4 


74 


87.0 


93.?. 


73 


83.5 


91.3 


72 


79.7 


90.0 


71 


76.1 


89.6 


70 


73.8 


87.9 


69 


70.6 


86.6 


68 


63.6 


85.9 


67 


56.5 


84.8 


66 


53.8 


81.8 


65 


49.1 


79.3 


64 


44.0 


76.0 


63 


43,0 


71.6 


62 


41.9 


69.0 


61 


39.1 


66.9 


60 


36.4 


64.6 


59 


33.7 


61.3 


58 


30.1 


58.6 


57, 


28.2 


55.2 


56 


27.5 


52.1 


55 


25.8 


49.4 


54 


23.7 


45.3 


53 


22.2 


39.7 


52 


18.2 


33.2 


51 


13.5 


29.4 


50 


11.2 


25.7 


49 


9.5 


22.1 


48 


9.1 


20.2 


47 


9.1 


16.6 


46 


9.1 


12.9 


45 


8.7 


10.4 


44 


8.3 


7.8 


43 


7.5 


6.0 


42 


6.7 


4.3 


41 


6.1 


2.7 


40 


5.2 


1.2 


39 


4.8 




38 


4.1 




37 


3.1 




36 


2.9 




35 


1.4 
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Table 5.11, continued 

Estimated National Noms for MLA Proficiency Test Skill Secret 
College Senior I^L Majors (Percentile Ranks of Converted Scores) 



Listening 


SPANISH 

Speaking 


Reading 


Writing 


Score 


%lle 


Score 


%lle 


Score 


%lle 


Score 


%ile 




Rank 




Rank 




Rank 




Rank 


56 


99.4 


118 


99.9 


69 


100.0 


77 


100.0 


55 


98.0 


117 


99.8 


68 


99.9 


76 


99.8 


54 


95.6 


116 


99.8 


67 


99.8 


75 


99.6 


53 


92.0 


115 


99.8 


66 


99.3 


74 


99.3 


52 


87.1 


114 


99.4 


65 


98.7 


73 


98.7 


51 


81.9 


113 


99.0 


64 


97.8 


72 


98.0 


50 


77.4 


112 


98.9 


63 


96.6 


71 


97.0 


49 


73.1 


111 


98.8 


62 


95.1 


70 


95.3 


48 


68.1 


110 


98.7 


61 


93.9 


69 


93.7 


47 


62.4 


109 


98.5 


60 


93.0 


68 


91.8 


46 


57.3 


lOG 


98.2 


59 


91.2 


67 


89.3 


45 


52.6 


107 


97.9 


58 


88.8 


66 


87.5 


44 


48.0 


106 


97.4 


57 


86.2 


65 


85.7 


43 


42.9 


105 


96.9 


56 


83.2 


64 


83.4 


42 


37.1 


104 


96.0 


55 


80.5 


63 


80.4 


41 


32.0 


103 


95.2 


54 


77.7 


62 


76.8 


40 


27.7 


102 


94.2 


53 


74.3 


61 


73.4 


39 


24.1 


101 


92.9 


52 


70.9 


60 


69.9 


38 


20.9 


100 


91.8 


51 


67.3 


59 


66.4 


37 


18.0 


99 


90.4 


50 


62.8 


58 


63.1 


36 


14.6 


98 


89.0 


49 


58.6 


57 


59.4 


35 


11.3 


97 


87.3 


48 


54.9 


56 


55.5 


34 


8.7 


96 


85.4 


47 


50.7 


55 


52.2 


33 


7.0 


95 


83.6 


46 


46.3 


54 


48.5 


32 


5.7 


94 


82.2 


45 


41.9 


53 


44.6 


31 


4.1 


93 


80.5 


44 


37.8 


52 


41.7 


30 


2.5 


92 


78.1 


43 


33.9 


51 


39.0 


29 


1.5 


91 


75.0 


42 


29.5 


50 


36.4 


28 


0.9 


90 


71.9 


41 


25.2 


49 


33.9 


27 


0.4 


89 


68.1 


40 


21.2 


48 


31.1 


26 


0.2 


88 


64.0 


39 


17.6 


47 


27.7 


25 


0.1 


87 


59.7 


38 


14.1 


46 


24.1 


24 


0.1 


86 


55.5 


37 


11.5 


45 


21.1 






85 


52.4 


36 


9.4 


44 


18.3 






84 


49.0 


35 


6.8 


43 


15.8 






83 


45.3 


34 


4.6 


42 


13.6 






82 


41.8 


33 


3.1 


41 


11.2 






81 


39.0 


32 


2.1 


40 


9.0 






80 


36.9 


31 


1.4 


39 


7.8 






79 


34.0 


30 


0.7 


38 


7.0 






78 


30.8 


29 


0.3 


37 


5.6 






77 


28.5 


28 


0.1 


36 


4.2 






76 


26.3 


27 


0.0 


35 


3.3 






75 


23.6 






34 


2.7 






74 


21.0 






33 


2.1 






73 


18.4 






32 


1.5 






72 


16.0 






31 


1.1 






71 


14.6 






30 


0.9 






70 


13.3 






29 


0.7 






69 


11.5 






28 


0.6 






68 


10.1 






27 


0.4 






67 


9.0 






26 


0.3 






66 


7.9 






25 


0.2 






65 


7.1 






24 


0.1 






64 


6.2 














63 


5.6 














62 


5.0 














61 


4.2 














60 


3.3 














59 


2.6 














58 


2.2 














57 


1.8 














56 


1.6 














55 


1.5 














54 


1.1 














53 


0.7 














52 


0.6 














51 


0.6 














50 


0.5 














49 


0.3 














48 


0.2 
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Table 5.12 (a) 

Estimated College Senior Major National Norms 
for Selected Percentile Ranks 



FRENCH (Est. N = 5564) 



Percentile 

fUinks 


Listening 


Converted 

Speaking 


Scores 

Reading 


Writing 


99 


56 


• 

97 


65 


68 


97 


55 


91 


64 


64 


95 


54 


89 


62 


62 


90 


53 


85 


60 


59 


85 


52 


82 


58 


57 


80 


51 


80 


56 


55 


75 


50 


79 


55 


53 


70 


49 


78 


53 


52 


65 


48 


77 


52 


50 


60 


46 


75 


51 


49 


55 


45 


74 


49 


47 


50 


44 


73 


48 


46 


45 


43 


to 

• 


47 


^ 45 


40 


41 


70 


46 


43 


35 


40 


69 


44 


42 


30 


38 


68 


43 


40 


25 


37 


66 


41 


39 


20 


35 


65 


40 




15 


33 


63 


38 


35 


10 


32 


61 


36 


33 


5 


30 


57 


33 


30 


3 


29 


55 


32 


28 


1 


27 


51 


29 


25 


No. cases actually 


(1162) 


(li23) 


(1194) 


(1193) 



tested 



Mean (est. norms) 

Standard Deviation 
(est. norms) 



43.08 

7.87 



72.75 

9.48 



47.86 

8.83 



46.05 

9.77 
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Table 5,12 (b) 

Estimated College Senior Major National Norms 
for Selected Percentile Ranks 





GERMAN 


(Est. N » 1848) 


. . . 




Percentile 

Ranks 


Listening 


Converted Scores 
Speaking Reading 


Writing 


99 


56 


112 


68 


74 


97 


56 


109 


67 


72 


95 


55 


107 


66 


71 


90 


53 


102 


. 62 


68 


85 


52 


100 


60 


65 


80 


51 


96 


59 


63 


75 


50 


95 


58 


62 


70 


49 


93 


56 


60 


65 


48 


91 


55 


58 


60 


47 


89 


54 


57 


55 


45 


88 


52 


54 


50 


44 


86 


50 


51 


45 


43 


85 


49 


48 


40 


40 


83 


48 


46 


35 


38 


81 


45 


44 


30 


36 


79 


44 


43 


25 


34 


78 


43 


40 


20 


33 


75 


41 


37 


15 


32 


72 


39 


35 


10 


30 


68 


36 


31 


5 


28 


65 


33 


27 


3 


28 


61 


32 


25 


1 


27 


48 


30 


24 


No. cases actually 
tested 

Mean (est. norms) 

Standard Deviation 
(est. norms) 


(337) 

42.55 

8.76 


(326) 

85.75 

12.91 


(336) 

49.85 

9.78 


(337) 

50.42 

13.51 



Table 5.12 (c) 

Estimated College Senior Major National Norms 
for Selected Percentile Ranks 



RUSSIAN . (Est. N » 481) 



Percentile 

Ranks 


Listening 


Converted 

Speaking 


Scores 

Reading 


Writing 


99 


55 


115 


64 


80 


97 


55 


111 


60 


79 


95 


53 


106 


59 


77 


90 


51 


99 


56 


75 


85 


49 


93 


51 


73 


80 


48 


88 


49 


72 


75 


47 


86 


48 


71 


70 


46 


84 


46 


69 


65 


45 


82 


44 


68 


60 


44 


81 


43 


67 


55 


44 


80 


41 


66 


50 


43 


79 


39 


65 


45 


42 


78 


38 


64 


40 


41 


77 


37 


61 


35 


40 


76 


36 


59 


30 


39 


74 


35 


58 


25 


37 


72 


34 


55 


20 


36 


69 


32 


52 


15 


35 


65 


31 


51 


10 


34 


60 


29 


49 


5 


33 


56 


26 


40 


3 


33 


53 


25 


37 


1 


32 


51 


24 


35 


No. cases actually 
tested 


(90) 


(89) 


(90) 


(88) 


Mean (est. norms) 


42.53 


79.24 


40.91 


62.54 


Standard Deviation 


6.03 


13.54 


9.69 


10.93 



(est. norms) 

a. 
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Table 5.12 (d) 

Estimated College Senior Major National Norms 
for Selected Percentile Ranks 



SPANISH (Est. N = 4439) 



Percentile 

Ranks 


Listening 


Converted 

Speaking 


Scores 

Reading 


Writing 




99 


56 


113 


65 


74 




97 


55 


105 


63 


71 




95 


54 


103 


62 


70 




90 


53 


99 


59 


67 




85 


52 


96 


57 


65 




80 


51 


93 


55 


63 




75 


49 


91 


53 


61 




70 


48 


90 


52 


60 




65 


47 


88 


50 


59 




60 


47 


87 


49 


57 




55 


46 


86 


48 


56 




50 


44 


84 


47 


54 




45 


43 


83 


46 


53 




40 


42 


81 


45 


51 




35 


42 


79 


43 


49 




30 


41 


78 


42 


48 




25 


39 


76 


41 


46 




20 


38 


74 


40 


45 




15 


36 


71 


38 


43 




10 


35 


68 


36 


40 




5 


32 


62 


34 


37 




3 


30 


60 


33 


35 




1 


28 


54 


30 


31 




No. cases actually 
tested 


(859) 


(820) 


(861) 


(865) 




Mean (est. norms) 


43.97 


83.52 


47.21 


53.80 




Standard- Deviation 
(est. norms) 


6.79 


12.02 


8.31 


10.16 
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PM 
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H 


C 
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<3 


h3 


a 


a 


h3 
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P4 


h3 
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Pi3 


Em 


Q 


Q 


Em 


Q 


Q 


Em 


Q 


Q 


Pm 


Q 
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a 


as 


as 
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as 


O 


as 


as 


O 


as 


PS 



'For the CFLT Data, the N given is the estimated number of students in the 1965 population. For the 
NDEA data, N*s are the average numbers taking the tests. 
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In a foreign language. The NDEA groups, on the other hand, were apparently composed of 
teachers who were very heterogeneous in background— ranging from the accomplished native 
speaker of the language to the person who has hardly had any training In the foreign lang- 
uage but was, say, transferred from English or science teaching to teach a foreign lang- 
uage. The differences between the CELT and the NDEA norms are particularly striking In 
the Speaking tests. In French and Russian, the NDEA post-test Speaking means are higher 
than the CELT means; In all four languages, the NDEA standard deviations are much higher 
than those In the CFLT data. 

Evaluation of CELT Norms against FSI Absolute Proficiency Ratings 

Certain conclusions about the CFLT data in relation to FSI "absolute language profic- 
iency" ratings can be drawn by Inspection of the figures (5.1 through 5.4). To provide a 
more exact basis for comparison, we have prepared Table 5,14, which shows the estimated 
percentages of the CFLT populations attaining or exceeding each FSI rating. 

. It Is Interesting to compare the language groups with respect to the proportions at- 
taining or exceeding FSI level "3" — "Minimum professional proficiency" as defined by the 
Department of State. This level may be regarded as a minimal qualifying level for a 
teacher of the foreign language. In French, 19. 3Z are qualified at this level in Listen- 
ing, only 4.4Z in Speaking, 47. 9Z in Reading, and 45. 7Z in Writing. In German, more are 
qualified In the audio-lingual skills than was the case In French, but about the same 
proportions In Reading and Writing. Specifically, the proportions qualifying In German 
in the I4LA Skills Tests are 47. 6Z in Listening, 22.1% in Speaking, 49. 8Z in Reading, and 
44. 7Z in Writing. 

Students of Russian are generally poor In comparison to FSI standards. Only 11. 3Z 
attain or exceed the "3" level in Listening, 6.2Z in Speaking, 17. OZ In Readltig, and 
12. 4Z In Writing. 

Students of Spanish show up reasonably well In comparison to the FSI standard; 33. 6Z 
qualify at the "3" level in Listening, 27. 8Z in Speaking, 58. IZ In Reading, and 47. 8Z in 
■ Writing. 

The reader will have noticed, no doubt, that over all languages, students are much 
less well prepared In Listening and Speaking skills than In Reading and Writing. 

Comparison of CFLT Norms against Myers >ielton Standards 

Some readers will prefer to evaluate the CFLT Norms with reference to the Myers- 
Melton (1964) qualification standards. Table 5.15 has been prepared to show the absolute 
numbers and percentages of the CFLT population falling in each of the Ifyers-Melton cate- 
gories, which, it will be recalled, were Intended to correspond to the qualification lev- 
els (Superior, Good, Minimal, and Unsatisfactory) established by a committee of the Modem 
Language Association In 1954. 

In general, the "Superior" level of the Myers-Melton standards is an even more dem- 
anding criterion than the "3" level of the FSI ratings. For example, in French, only 
16.48Z of the population is estimated to qualify as "Superior" in Listening, whereas 19. 3Z 
qualify at the "3" level of the FSI ratings. Less than IZ of the French students qualify 
as Superior in Speaking, ss'^d only 25.20Z in Reading and 7.56Z in Writing. Similar com- 
parisons may be made for the other language groups. 

On the other hand, according to the Ifyers-Melton standards, relatively few of the 
CFLT population fall into the Unsatisfactory category. (In Russlap, the Jtyers-Melton rat- 
ings do not distinguish between the Minimal and Unsatisfactory ratings.) 

Still, whether one uses the FSI ratings or the Myers-Melton ratings as standards, the 
net showing of the CFLT population Is not good. The CFLT groups are particularly defici- 
ent In audio-lingual skills. Only about a third to a half are at a satisfactory level 
even Ip Reading and Writing skills. 







Table 5.14 



Estimated Percentages of CELT Population 
Attaining or Exceeding Each FSI Rating 



FSI 

tig" 


Listening 


Speaking 


FSI 

"R" 


Reading 


Writing 






FRENCH 






S-5 


ic 


0.0 


R-5 


* 


0.0 


S-4+ 


•k 


0.0 


R-4+ 


0.5 


1.7 


S-4 


k 


0.0 


R-4 


8.0 


9.7 


S-3+ 


1.3 


1.1 


R-3+ 


25.4 


24.2 


S-3 


19.3 


4.4 


R-3 


47.9 


45.7 


S-2+ 


40.4 


17.5 


R-2+ 


70.4 


68.0 


S-2 


58.8 


47.5 


R-2 


87.5 


86.1 


S-1+ 


• 74.4 


76.2 


R-1+ 


98.1 


96.3 


S-1 


88.2 


94.8 


R-1 


100.0 


99.8 






GERMAN 




1 


S-5 


k 


* 


R-5 


* 


* 


S-4+ 


2.0 


0.0 


R-4+ 


5.1 


1.7 


S-4 


16.9 


0.4 


R-4 


13.2 


11.0 


S-3+ 


34.0 


6.1 


R-3+ 


32.1 


28.4 


S-3 


47.6 


22.1 


R-3 


49.8 


44.7 


S-2+ 


58.5 


50.0 


R-2+ 


65.2 


56.1 


S-2 


67.0 


75.9 


R-2 


80.9 


72.7 


S-1+ 


77.0 


89.6 


R-1+ 


90.2 


85.2 


S-1 


89.5 


98.6 


R-1 


98.4 


94.4 






RUSSIAN 






S-5 


* 


* 


R-5 


* 


* 


Cl 

S-4+ 


* 


* 


R-4+ 


* 


* 


S-4 


fc 


0.0 


R-4 


0.0 


* 


S-3+ 


3.4 


2.4 


R-3+ 


6.4 


* 


S-3 


11.3 


6.2 


R-3 


17.0 


^ 12.4 


S-2+ 


27.5 


14.8 


R-2+ 


39.8 


46.0 


S-2 


55.7 


27.3 


R-2 


71.3 


69.9 


S-1+ 


72.0 


53.5 


R-1+ 


94.0 


89.0 


S-1 


93.8 


80.4 


R-1 


100.0 


93.4 






SPANISH 






S-5 


* 


0.2 


R-5 


3.3 


* 


S-4+ 


* 


1.6 


R-4+ 


9.8 


0.0 


• S-4 


1.6 


5.2 


R-4 


22.0 


2.6 


S-3+ 


15.5 


13.9 


R-3+ 


39.3 


19.0 


S-3 


33.6 


27.8 


R-3 


58.1 


47.8 


S-2+ 


52.5 


51.7 


R-2+ 


76.8 


73.0 


S-2 


71.8 


70.4 


R-2 


90.8 


92.4 


S-1+ 


84.0 


85.1 


R-1+ 


98.4 


99.0 


S-1 . 


93.5 


93.0 


R-1 


100.0 


ino.o 



*The FSI rati.ng equivalent exceeds the maximum score on the test 



O 

ERIC 



■•■••.sv 



- 100 - 



Table 5.15 

Estimated Numbers and Percentages of CFLT Population 
Falling in Each of the Myers-Melton Qualification Categories 

Listening Speaking Reading Writing 

N% N% N% N% 



FRENCH 



Superior 


917 


16.48 


18 


0.33 


1402 


25.20 


420 


7.56 


Good 


2758 


49.56 


1237 


22.23 


2716 


48.81 


2639 


47.43 


Minimal 


1347 


24.20 


4002 


71.92 


1376 


24.73 


2359 


42.40 


Unsatii^factory 


543 


9.76 


29 


0.52 


70 


1.26 


145 


2.61 


Total 


5565 


100.00 


5565 


100.00 


5565 


100.00 


5565 


IQO.OO 








GERMAN 










Superior 


692 


37.45 


70 


3.78 


458 


24.78 


364 


19.72 


Good 


651 


35.21 


792 


42.84 


1001 


54.17 


826 


44.70 


Minimal 


391 


21.16 


784 


42.41 


266 


14.38 


524 


28.36 


Unsatisfactory 


114 


6.18 


203 


10.97 


123 


6.67 


133 


7.22 


Total 


1848 


100.00 


1848 


100.00 


1848 


100.00 


1848 


100.00 








RUSSIAN 










Superior 


129 


26.76 


73 


15.14 


225 


46.83 


131 


27.20 


Good 


269 


55.94 


149 


31.10 


187 


38.89 


310 


64.52 


Minimal 


83 


17.30 


258 


53.76 


69 


14.28 


40 


8.28 


Unsatisfactory 


















Total 


481 


100.00, 


481 


100.00 


481 


100.00 


481 


100.00 








SPANISH 










Superior 


1532 


34.52 


1039 


23.40 


. 1213 


27.33 


590 


13.29 


Good 


1913 


43.10 


2413 


54.35 


2363 


53.23 


2539 


57.20 


Minimal 


851 


19.18 


897 


20.21 


789 


17.77 


1204 


27.12 


Unsatisfactory 


142 


3.20 


91 


2.04 


74 


1.67 


106 


2.39 


Total 


4439 


100.00 


4439 


100.00 


4439 


100.00 


4439 


100.00 
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Qiapter VI 



STUDENT FACTORS IN FOREIGN LANGUAGE SKILL ATTAINMENT 



1. Introduction 



"Student factors" are those characteristics of students that they bring with then 
to the Instructional program; some of these characteristics may be Inherent In them, 
like sex and age, while others, like motivations and Interests, may be modified by 
their experiences In the course of foreign language study. Since this Investigation was 
a descriptive survey In which all measurements and observations were taken at roughly 
a single point of time In the life histories of our subjects. It Is Impossible to draw 
well-founded Inferences as to any causal relationships that may exist between the 
characteristics of students and their foreign language attainments at the time of grad- 
uating from college. We cannot even indicate what changes may have occurred In student 
characteristics in the course of time. All we can do is to compare different groups 
of students with respect to foreign language attainments and observe any statistical 
associations that may exist. Nevertheless, it is of Interest to describe different 
groups, note any differences In foreign language attainments, and attempt to suggest 
possible reasons for these differences. In any case, the findings are likely to throw 
light on the interpretation of test scores. For example, if there were marked sex 
differences In foreign language attainments (actually, there are not) a given score 
might have a different interpretation depending on the sex of the Individual. The find- 
ings would also be useful In analyzing the variations that may exist In the sample of 
graduating foreign language majors studied her^. 

All analyses reported In this chapter are based on the actual sample of foreign 
language majors that were tested, or upon sub-samples thereof. No effort Is made to 
estimate the characteristics of students who were not tested, or of the total population 
of students tested and not tested). The limitations of the sample pointed out In 
Chapters III and V must therefore be borne in mind in interpreting the findings. 

For the most part, fairly standard statistical techniques have been employed to 
make the analyses. The reader who wishes to learn more about these techniques should 
consult standard textbooks of statistics, such as McNemar (1962) or Hays (1963). Many 
of the analyses in this and subsequent chapters were performed on high-speed computers 
and employed programs reported by Cooley and Lohnes (1962) and Jones (1964). 

0 

0 



2. Sex and Age 



The analyses to be presented here are based on all seniors complete on the four 
MLA skills tests and on pages 1 and 4 of the Student Questionnaire giving background in- 
formation, and who are "regular cases" who had a normal course of training in foreign 
languages starting either in elementary school, high school, or college. Data for 
Italian were generally not analyzed because of the small numbers of cases. As far as 
can be determined, there is little bias in the sample with respect to sex (See Chapter 
III). As was pointed out earlier, the majority of foreign language majors are femalis; 
in the particular sample used here, males constitute only 15. 9Z of the French majors, 

41. OZ of the German maj9*“5, 37.5% of the Russian majors, and 24.8% of the Spanish majors. 

Males' and females' score distributions on each skill test in eacho language were 
compared by means of the ^-test, with results shown in Table 6.1. Only one difference 



<5 



Table 6.1 



MLA Skills Scores by Sex 



Skill 




Males 






Females 




Total 




t 


N • 


Mean 


S.D. 


N 


Mean 


S.D. 


N 


Mean 


S.D. 








FRENCH 














Listening 


177 


43.8 


8.4 


937 


44.2 


7.5 


1114 


44.1 


7.6 


-0.68 


Speaking 


If 


74.6 


10.2 


ff 


73.4 


9.0 


ff 


73.6 


9.2 


1.62 


Reading 


If 


49.2 


10.0 


ff 


48.7 


8.3 


ff 


48.8 


8.6 


0.60 


Writing 


If 


46.2 


10.7 


ff 


47.3 


9.3 


ff 


47.2 


9.5 


-1.49 








GERMAN 














Listening 


133 


43.3 


8.5 


191 


43.5 


8.7 


324 


43.4 


8.6 


-0.17 


Speaking 


If 


87.4 


12,2 


ff 


86.3 


13.2 


ff 


86.8 


12.8 


0.80 


Reading 


If 


51.4 


10.1 


ff 


50.8 


9.5 


ff 


51.0 


9.8 


0.55 


Writing 


If 


52.4 


13.1 


ff 


51.8 


13.5 


ff 


52.1 


13.3 


0.39 








RUSSIAN 














Listening 


33 


41.7 


6.0 


55 


43.1 


5.9 


88 


42.6 


6.0 


-1.07 


Speaking 


If 


78.9 


14.2 


ff 


78.0 


11.6 


ff 


78.3 


12.7 


0.32 


Reading 


If 


40.1 


8.7 


ff 


42.0 


10.1 


ft 


41.3 


9.6 


-0.85 


Writing 


If 


61.9 


11.3 


ff 


62.2 


10.7 


ff 


62.1 


10.9 


-0.11 








SPANISH 














Listening 


202 


44.8 


10.3 


612 


44.5 


6.7 


814 


44.5 


6.8 


0.37 


Speaking 


If 


86.0 


12.3 


ff 


83.4 


11.8 


If 


84.0 


12.0 


2.74-fc* 


Reading 


If 


48.0 


8.6 


ff 


47.7 


8.3 


ff 


47.8 


8.4 


0.41 


Writing 


ft 


54.2 


10.8 


ff 


54.8 


9.8 


ff 


54.6 


10.1 


-0.72 



**Significance (p < .01) 
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was significant beyond the 5Z level: in the Spanish group, males were significantly 
superior in Speaking, at the IX level of significance (two-tailed test). We have no 
explanation for this and since all the other differences were only what could be expect- 
ed by chance, it may be regarded as a "chance" significant difference. There was not 
even a pattern over languages to suggest that one sex tended to be superior in some 
skills but not in others. As far as our sample is concerned, males and females do not 
differ in average foreign language skill attalnL.ent. 

The students were asked to state their "age." (No specification such as "age at 
nearest birthday" or "age at last birthday" was asked for in the questionnaire.) Table 
6.2 shows the distribution of ages stated, by language. The modal age in all languages 
is 21; 88. 8Z of the ages are comprised within 20, 21, and 22. The distributions are 

nevertheless highly skewed; while there is only a scattering of ages 18 and 19, there 
are substantial numbers of students graduating in the late 20* s, the 30* s, and 40 s. 

In each language, the group was broken down into three subgroups, 20 or below, 21 and 22, 
and 23 and over, and these subgroups were compared by one-way analysis of variance with 
respect to their mean MLA skills scores, with results as shown in Table 6.3. Many of 
these comparisons approached significance at the 1% level, but there are no clear 
patterns. In French and Spanish there is a slight tendency for the younger graduates to 
have auperior scores. This may possibly foreshadow the finding (Chapter VII) that those 
who started foreign language study early had higher scores than those who started later. 
In any case, the age groups are in general so close together in ^^ttalnment that no 
recommendations concerning this variable can be given. There is certainly no strong 
tendency for older students to obtain inferior scores. 

There is a slight tendency (the data are not shown in the tables) for dispropor- 
tionately more of the older students (23 and over) to be males. 



Students tested in the national study are on the average quite high on both the 
part and the total scores of the Modem Language Aptitude Te st (MLAT) . Table 6.4 shows 
the means and standard deviations of these scores for four language groups, as well as 
the limited normative data available from the Manual (1959) of the MLAT . There are 
three possible factors making for superior average scores for the CFLT samples: 

(1) Norms shown in the manual for high school (Grades 9 , 10, 11 and college freshman 
groups exhibit monotonically increasing mean scores. For example, "short form' (total, 
parts 3-5) means for females for fairly large norm groups are as follows: 



Cn the assumption that growth continues throughout the college years, it is Ui5t unrea- 
sonable to expect average scores for seniors to be somewhat higher than the College 
Freshmen norms. As may be seen in Table ,6.4, the distribution of MLAT Total scores for 
all 2364 cases in the CFLT sample has a mean of 76.2 and a standard deviatic’i of 17.4. 

(2) The norms shown in the Manual and cited above are for unselected students taking 
foreign language courses. It could be expected that students majoring in foreign 



3. Foreign Language Aptitude 



College Freshmen 



Grade 9 
Grade 10 
Grade 11 



N Mean S.D. 
391 43.3 13.2 
304 46.8 15.6 
233 53.4 14.5 
277 61.4 17.3 



o 
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Table 6,2 



Age Distributions of FL Majors at Graduation, 

by Language 



LANGUAGE 





FRENCH 


GERMAN 


ITALIAN 


RUSSIAN 


SPANISH 




TOTAL 


Age 


f 


cum. 

% 


f 


cum. 

% 


f 


cum. 

% 


f 


cum. 

% 


f 


cum. 

% 


f 


cum. 

% 


18 


0 


0.0 


0 


0.0 


0 


0.0 


0 


0,0 


1 


0.1 


1 


0.04 


19 


7 


0.6 


0 


0.0 


0 


0.0 


0 


0.0 


5 


0.7 


12 


0.5 


20 


81 


7.9 


15 


4.6 


1 


4.2 


8 


9.1 


51 


7.0 


156 


7.1 


21 


686 


69.5 


175 


58.6 


18 


79.2 


45 


60.2 


448 


62.0 


1372 


65.2 


22 


256 


92.5 


88 


85.8 


3 


91.7 


25 


88.6 


201 


86.7 


573 


89.4 


23 


30 


95.2 


14 


90.1 


0 


91.7 


2 


90.9 


37 


91.3 


83 


92.9 


24 


15 


96.5 


11 


93.5 


0 


91.7 


3 


94.3 


19 


93.6 


48 


95.0 


25 


11 


97.5 


8 


96.0 


0 


91.7 


0 


94.3 


11 


95.0 


30 


96.2 


26 


3 


97.7 


5 


97.5 


0 


91.7 


3 


97.7 


8 


95.9 


19 


97.0 


27 


2 


97.9 


3 


98.5 


1 


95.8 


0 


97.7 


8 


96.9 


14 


97.6 


28 


2 


98.1 


2 


99.1 


0 


95.8 


0 


97.7 


5 


97.5 


9 


98.0 


29 


2 


98.3 


1 


99.3 


0 


95.8 


0 


97.7 


3 


97.9 


6 


98.3 


30-34 


2 


98.5 


1 


99.7 


1 


100.0 


0 


97.7 


8 


98.9 


12 


98.8 


35-39 


6 


99.0 


0 


99.7 


0 




0 


97.7 


2 


99.1 


8 


99.1 


40-44 


8 


99.7 


1 


100.0 


0 




1 


98.9 


4 


99.6 


14 


99.7 


45-49 


2 


99.9 


0 




G 




0 


98.9 


1 


99.8 


3 


99.8 


50+ 


1 


100.0 


0 




0 




1 


100.0 


2 


100.0 


4 


100.0 


Total 


1114 


324 




24 




88 




814 




2364 




Table 6.3 



Mean MLA Skill Test Scores by Age Groups 



Language 


Age Group 


N 


Listening 


Speaking 


Reading 


Writing 



FBJENCH 


20 or less 


88 


46.4 


76.2 


50.2 


50.0 




21-22 


942 


43.9 


73.2 


48.6 


46.9 




23+ 


84 


43.7 


74.3 


49.6 


47.0 




Total 


1114 












P 




<.02 


<.02 


.18 


<.02 


GERMAN 


20 or less 


15 


42.9 


88.3 


52.0 


55.1 




21-22 


263 


43.0 


85.8 


50.6 


51.5 




23+ 


46 


46.3 


91.7 


53.0 


54.6 




Total 


324 












P 




.06 


<.02 


>.10 


>.10 


RUSSIAN 


22 or less 


78 


42.2 


77.5 


41.0 


61.6 




23+ 


10 


45.1 


84.9 


43.1 


65.9 




Total 


88 












P 




>.10 


.08 


>.10 


>.10 


SPANISH 


20 or less 


57 


46.5 


82.2 


50.0 


58.1 




21-22 


649 


44.3 


83.7 


47.4 


54.2 




23+ 


108 


44.6 


86.7 


49.0 


55.0 




Total 


814 
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Table 6.4 



Modem Language Aptitude Test scocea, 
by language, as compared 
with norms 

Samples: All tested students complete on MLA Skill 

Scores and Parts 3, 4, & 5 of MLAT, 
excluding non-seniors and Italian majors 



Language 


N 




National CFLT data 
MLAT 

Pt. 3 Pt. 4 Pt. 5 


Total 

3-5 


Norms 

N 


from Manual’^ 


FRENCH 


1084 


M: 


27.4 


32.1 


19.6 


79.1 


55 


M: 


61.7 






SD: 


9.9 


6.5 


5.0 


15.9 


(Women) 


SD: 


16.1 


GERMAN 


349 


M: 


24.1 


31.3 


18.8 


74.2 


32 


M: 


54.6 






SD: 


9.9 


7.6 


5.8 


18.1 


(Women) 


SD: 


19.5 


RUSSIAN 


94 


M: 


28.5 


32.0 


19«6 


80.1 


73 


M: 


68.9 






SD: 


12.0 


7.9 


5.6 


20.4 


(Men) 


SD: 


19.1 


SPANISH 


837 


M: 


24.7 


29.7 


18.6 


73.0 


39 


M: 


57.8 






SD: 


10.0 


7.8 


5.7 


17.8 


(Women) 


SD: 


16.2 


TOTAL 


2364 


M: 


26.0 


31.1 


19.1 


76.2 












SD: 


10.1 


7.3 


5.4 


17.4 









*These norms are from data shown for "College Sophomores, Juniors, and Seniors" on 
the "Short Form" of the MLAT, from the Manual , p. 11. The "Short Form" Is the same as 
Parts 3-5 and therefore these data are to be compared with the total scores on Parts 
3-5 for the CFLT samples. 




%■ 
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IsnguAgss might have higher average scores than samples of students who are not necessar** 
ily ''majoring'“ in a foreign language. 

(3) It is possible that foreign language study as such teaches certain skills and 
knowledges tested by the MLAT and therefore that persons with large amounts of foreign 
language study will attain higher scores than persons with lesser amounts of such study. 



The present study offers no direct evidence as to the magnitude of any of t^ ese 
factors. Chapter VII will offer some indirect evidence that suggests that the third of 
these factors has little effect* it will be shown* that is* that persons with different 
amounts of foreign language study do not differ much* on the average* in MLAT scores. 

Another interesting aspect of Table 6.4 is the comparison of average MLAT scores 
across languages. The mean total scores rank as follows: 

Russian 80.1 

French 79.1 

German 74.2 

Spanish 73.0 

jljn analysis of variance of these data yields an F— ratio that is highly significant. The 
results tend to agree with results obtained elsewhere in research on the MLAT : those 

who select a "difficult” language like Russian tend to have higher foreign language 
aptitude* and those who select an allegedly "easy” language like Spanish tend to have 
lower aptitude scores. Apparently there is some degree of self— selection of languages 
according to language aptitude; it is known from previous research (Carroll* 1932) 
that individuals' ratings of their own language aptitude tend to correlate with their 
scores on the MLAT and therefore it is not unreasonable to think that students can and 
do select languages to study in accordance with their own Images of their language 
aptitude. 

Although the average MLAT scores of the CFLT group are high* the score distributions 
show considerable range* as indicated by the standard deviations i which are comparable 
with those of the distributions of normlng groups discussed in the MLAT manual. The 
question can be raised* now* whether language aptitude as measured by the shows any 

significant relationship with foreign language attainment. A subsidiary but related 
question is whether the several parts of the MLAT— or at least the three parts used in 
this study— measure different aspects of language aptitude. Relevant data are shown in 

Table 6.5. 

The intercor relations of parts of the MLAT are relatively low; this finding accords 
with results of previous research (Carroll and Sapon* 1958). The parts do Indeed measure 
different aspects of language aptitude. 

The correlations of MLAT scores with MLA skill test scores are generally very low; 
some of them are negative. Only in the case of the French and Spanish grouT>s are they 
generally positive* but even then* they barely reach statistical significance. These 
findings* taken at face value* would seem to show that language aptitude has very little 
to do with success in learning a language. It would be hasty to draw such a conclusion* 
however* for It must be remembered that the groups studied here are very heterogeneous in 
their backgrounds. From previous research comes the clear suggestion that language 
aptitude is relatCf^. to the amount of time the individual will have to take to acquire a 
specified degree of competence. These groups are very heterogeneous in the amount of 
time they have studied their foreign languages; as will be shown in Chapter VII* the 
cases vary dramatically with respect to how early in their student careers they started 
studying the foreign languages in which they are majoring* and also with respect to the 
amount of opportunity they have had to acquire competence through foreign travel and 
study and through exposure to the foreign language at home. It is only after all these 
factors are controlled* by statistical means* that it will be possible to examine the 
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Table 6.5 

Intercorrelations of MLAT Part Scores and Correlations 

with MLA Skill Scores 

(Same samples as In Table 6.4) 



FRENCH (N * 1084) 





Part 3 


Part 4 


^art 5 


MLAT Part 3 


1.00 


.30 


.24 


(Spelling Clues) 








MLAT Part 4 


.30 


1.00 


.35 


(Words In Sentences) 








MLAT Part 5 


.24 


.35 


1.00 


(Paired Associates) 








MLAT Total, Tarts 3-5 


.82 


.71 


.61 






GERMAN 


(N = 349) 




Part 3 


Part 4 


Part 5 


MLAT Part 3 


1.00 


.43 


.32 


MLAT Part 4 


.43 


1.00 


.44 


MLAT Part 5 


.32 


.44 


1.00 


MLAT Total, Parts 3-5 


.83 


.79 


.68 






RUSSIAN 


(N = 94) 




Part 3 


Part 4 


Part 5 


MLAT Part 3 


1.00 


.42 


.48 


MLAT Part 4 


.42 


1.00 


.37 


MLAT Part 5 


.48 


.37 


1.00 


MLAT Total, Parts 3-5 


.88 


.74 


.70 






SPANISH 


(N = 837) 




Part 3 


Part 4 


Part 5 


MLAT Part 3 


1.00 


.38 


.26 


MIAT Part 4 


.38 


1.00 


.38 


MLAT Part 5 


.26 


.38 


1.00 


MLAT Total, Parts 3-5 


.81 


.77 


.63 



MLA Skill Score 



Listening 


Speaking 


Reading 


Writing 


.20 


.16 


.25 


.23 


.11 


.05 


.22 


.25 


.10 


.01 


.16 


.14 


.21 


.13 


.30 


.29 



Listening 


Speaking 


Reading 


Writing 


-.08 


-.13 


-.06 


-.07 


-.12 


-.23 


-.04 


.02 


-.14 


-.26 


-.16 


-.16 


-.14 ' 


-.25 


-.10 


-.08 



Listening 


Speaking 


Reading 


Writing 


-.15 


-.26 


-.19 


-.20 


.06 


• -.01 


.07 


.06 


-.13 


-.20 


-.16 


-.14 


-.10 


-.21 


-.12 


-.13 



Listening 


Speaking 


Reading 


Writing 


.14 


.07 


.15 


.20 


.00 


-.08 


.02 


.07 


.00 


-.06 


.03 


.04 


.08 


-.02 


.10 


.16 



I 



o 
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true relationships between language aptitude and the attainment of foreign language 
skill. This is attempted in Chapter VIII, and it will be shown there that language 
aptitude has significant positive relationships to skill attainment when the effects 
of extraneous variables are eliminated. 



4. Plans. Interests, and Motives in Foreign Language Study 



In the student questionnaire, the following question appeared: 

1) For what reason(s) did you decide to concentrate in the languag^ in which you are being tested? 
Check as. many as apply: 

C ] I expect to teach a foreign language or languages. If so, check the level or levels at 
which you plan to teach: 

[ '] at the elementary school level 
[ ] at the secondary school level 
[ ] a college, university, or graduate school 
[ ] other (specify) 

[ ] I expect to use my foreign language knowledge in my future employment or profession. If 
so, specify the kinds of work you are contemplating: 

[ 3 Linguistics 

[ J Business activity (importing, foreign trade, etc.) 

[ 3 Government work (foreign service. United Nations, A.I.D., fete.) 

[ 3 Military service 
[ 3 Work as a translator or interpreter 

[ 3 Other professional work (medicine, science, library work, etc.) 

[ 3 I expect to use my foreign language knowledge in travel or study abroad. 

[ 3 I majored in a foreign language mainly for interest and enjoyment. 

[ 3 I majored in a foreign language because I am particularly interested in the literary 
and cultural aspects of the language. 

■ 3 I am interested in intercultural understanding and communication. 

’ 3 No particular reason other than the fact that I needed a major subject. 

[ 3 Other reasons (specify) • 



The responses of 2340 French, German, Russian, and Spanish majors whose cases were com- 
plete on MLA skill test scores are tabulated in Table 6.6. The results for the major 
categories depict the main reasons why the students thought they were majoring in a 
foreign language. 

Because of the structure of the question, which permitted various combinations of 
multiple responses, it is not possible to set up exclusive categories for students' 
reasons for majoring in a foreign language. About half of those planning to use the 
foreign language In future employment also indicated that they planned to teach. 



Overall, 64.6% of the majors plan to go into the teaching of foreign languages, at 
some level. A slightly higher proportion of females (65.6%) are interested in teaching 
than are males (61.1%) The four languages are more or less equal in this respect, Spanish 
being the language where a higher rroportion (67.6% are attracted to teaching, and Russian 
being the language where the smallest proportion (51.1%) are attracted to teaching. 

Russian is also different from the other languages in that more males than females plan 
to teach it. 

Table 6.7 shows a detailed analysis of the groups planning teaching. Teaching at 
the secondary school level draws by far the largest percentage of responses — 85.3%; 
females (89.3%) more than males (71.2%). The languages rank as follows: Spanish (89.4%), 

French (86.0%), German (78.0%), Russian (55.6%). Relatively few males (27.8%) plan 
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teaching Russi®. at this level, hwever. teathrii'atKacir^^ 

:rt»r <3"«J - jirsW^ 

‘rrte*:;se^!“»d"Lft ff%he“ are’^fenales planning to teach either French or Spanish. 

Table 6.7 also shows the P*®P"‘^“®i^°“jomairrthrt^^ "qualify' for a 

in Question 3 on Page 4 of the Student These percentages closely paral- 

UcLe or certificate "faeitag^^^^^^ to tLch at the seconda^ 

s:i;ori:«l? “vw^ur-st of 'those Pl^^J,^%*“'ach?ng'c«^^^^^^^ “a^d'there are ,, 
“aifo ::herrwSo%rerthtra”e:X'c:rtifica^ would be a good thing to have "in case. 

Table 6.8 gives an analysis 

foreign language skills in work oth .„ssian)^ and almost as many plan on work in 

on government work f sur^rtsing proportion (39.0T) think of work in 

translating or interpreting, P“ticu J.y f ^ J^^ar professional work such as in 

Returning to Table 6.6, we see »* 

in the "fringe benefits" of foreign f majored in a foreign language because 

abroad (71.8%). 60.8% f.‘*®a'Sndini1^SlcS wi^^^^^ Pl®®*® language 

of "interest and enjoyment --a «“ding "h literary and cultural aspects of 

teacher. Roughly half qi^’clalm interest in the implications of foreign language 

foreign language study, and 58.« cl^m inte „portlons take foreign language 

study for intercultural “»f”*®”""ii-f,^’^hn « ^ conclude that nearly all the 

Another aspect of imterest ^“^“fs/usLnLg^'spe^^ raiding, md 

portance attached to P^®/““',*’®®J? ®o I'e related in considerable degree to their 
writing. Students’ opinions in this area ^ foreign language. To make analy- 

vocatlonal plans and to other *®®® gjujents were pooled over foreign langu^e 

— : ;“vi:n|hs; 

ii»p a. I” sl^ ond of the r^^ction entitled By Intent- 
Over all students (see data for ° ® rated as ”of great Importance,” On the 

ions to Teach”) all four ortance, and Listening was a clo&e second, 

average Speaking was rated as of most imports , oriting had the lowest average im- 
ReadiSg had the third ^J^hest i^mportance L-ange of "great Import- 

rt“lreir\u'fo«^Wll»“«®“ iated higher by those planning to teach than 
by those not planning to teach. 

Those planning to teach only at ®^® ®^®^®"‘“2ipg*pJ^ii||^to'te^^^ at^other levels. 

lower Importance ratings ®“ lower Importance ratings to Reading and "**‘^"8- 

They were particularly inclined to glv shown below) that those planning to 

This finding possibly reflects the fa ( jnj^ ,fciii test scores thm 

teach at the elementary ^®'*®} *® ' . levels. Furthermore, Importance ratings 

rd%l^-ellar:frrS^^^^^^^ scores; on this point, see below. 

Ratings of the i”P®'‘®^“ "ollegl^llvel^ md'thlse'are the students who tend to have 
riilirrr'agl'SJl :lm: :-:i: as Sh;wn m Tabu 6.U. 
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Table 6.9 

Mean Importance Ratings Given to Foreign Language Skills by Students 
Sample: 2340 FL Majors ("tegular Cases") In All Languages Except Italian 



Coding of Scale: 3 « great Importance 

2 ■■ some Importance 
1 * little Importance 



Group 


N 


Z 


Listen. 


Speak. 


Read. 


Write 


By Intentions to Teach 
Planning to Teach: 














-At Elementary Level Only 


59 


2.52 


2.vSl 


2.93 


2.58 


2.42 


-At Elem. or Sec. Levels 


99 


4.23 


2.95 


2.97 


2.86 


2.74 


-At Secondary Level Only 


964 


41.20 


2.90 


2.97 


2.80 


2.71 


-At Sec. or College Level 


196 


8.38 


2.92 


2.95 


2.90 


2.80 


-At College L^vel Only 


270 


11.54 


2.91 


2.95 


2.96 


2.82 


-Other level, or no level given 


64 


2.74 


2.91 


2.95 


2.82 


2,72 


All Planning to Teach 


1652 


70.60 


2.90 


2.96 


2.83 


2.73 


All Not Planning to Teach 


688 


29.40 


2.82 


2.77 


2.74 


2.49 


Total 


2340 


100.00 


2.88 


2.91 


2.81 


2.66 


By vocational goals 














All Planning to Use FL In 

Future Employment (other than teaching) 863 


36.88 


2.91 


2.89 


2.79 


2.65 


-In Government Work 


496 


21.20 


2.94 


2.92 


2.79 


2.66 


-In Business 


368 


15.73 


2.92 


2.90 


2.75 


2.65 


-In Translating or Interpreting 


337 


14.40 


2.94 


2.94 


2.84 


2.75 


-In Linguistics 


139 


5.94 


2.91 


2.92 


2.85 


2.75 


-In Military Service 


55 


2.35 


2.95 


2.91 


2.69 


2.55 


-In Other Professional Work 


168 


7.18 


2.81 


2.76 


2.79 


2.51 


By Other Reasons for FL Study 














Use FL In travel 


1679 


71.75 


2.92 


2.94 


2.82 


2.69 


"Interest and Enjoyment" 


1423 


60.81 


2.89 


2.90 


2.82 


2.67 


Interest In foreign culture 


1166 


49.83 


2.90 


2.90 


2.88 


2.72 


Interested In intercultural 
understanding 


1364 


58.29 


2.93 


2.93 


2.83 


2.70 


"Needed a Major" 


58 


2.48 


2.67 


2.72 


2.60 


2.33 




Students planning to use their foreign language skills In some future employment 
other than teaching (whether or not they In addition plan to teach) tend to give slightly 
higher Importance ratings to Listening, and slightly lower ratings to the other three 
skills, as compared to the total group. However, It Is among those planning to use 
foreign language knowledge In translating or interpreting, or In linguistics, that Read- 
Ing and Writing attract the highest Importance ratings. 

All these findings give strong support to contemporary policies In foreign language 
teaching whereby listening and speaking are accorded emphasis and attention at least 
equal to that accorded to reading and writing. Most students seem to regard reading and 
writing as "advanced" skills of great importance, particularly for specialized work such 
as college teaching, translating and Interpreting, and linguistics, but listening and 
speaking are regarded as being of even greater Importance, particularly for teaching at 
elementary and secondary school levels. 

Nevertheless, students do differ In the relative Importance they attach to listening 
and speaking, on the one hand, and reading and writing, on the other. In Table 6.10, 
where (among other things) the Intercorrelations of Importance ratings are given for four 
language groups, we see that Importance ratings of listening and speaking, and also the 
Importance ratings of reading and writing, are much more highly corr' »ated with each 
other than with the other pair of skills. Further, the language gc differ somewhat 
in their mv3an ratings. French, German, and Spanish groups characteristically tend to 
attach more Importance to listening and speaking than to reading and writing (the con- 
trast is particularly strong In Spanish) , while the Russian group attaches most import- 
ance to reading, less to listening and speaking, and least to writing. These differences 
reflect well-known facts about the use of these languages: American students In French 
and German are Interested about equally In oral communication with speakers of those 
languages and in their literatures; students of Spanish are Interested more In oral com- 
munication than In Spanish literature, and students of Russian are more Interested In 
reading literature (either belles-lettres, or scientific writings, or historical and 
political literature) than they are in oral communication with speakers of Russian. 



Let us re— emphasize at this point , however , that even within language groups students 
differ In the relative Importance they attach to oral and to written skills. To what 
extent are these differences related to their attainments in these skills as measured 
by the MLA skll7j tests? Table 6.10 contains also the correlations of Importance ratings 
with MLA skills scores. First we note that all these correlations are positive, indi- 
cating that the general degree of Importance a student attaches to any foreign language 
skill is somewhat related to his foreign language attainments in general. We might 
expect also, however, that the student who attaches greater Importance to oral skills 
will have made greater attainment in these skills, and likewise, the student who attaches 
greater Importance to written language skills will do better, relatively. In reading and 
writing tests. Table 6.11 presents a special analysis that sought to Investigate this 
matter. It utilizes a rather rarely employed statistical technique known as canonical 
analysis. (For a fairly detailed explanation of this technique, see Chapter VIII, 

Section 4.) The results are clearest in French, where there was thfe largest number of 
cases. The first canonical correlation for that group, .24, was highly significant and 
provides a measure of the fact that (as indicated by the canonical regression weights) 
there Is generally a correlation between Importance ratings and skills test scores. That 
Is, If we were to add up the Importance ratings, with all positive weights, we could 
predict a linear combination of test scores— a linear combination that would weight the 
speaking and writing skill scores. This finding merely confirms, with precision, the 
observation made above that the Importance ratings and skills scores are all positively 
corrvclated, and It tends to confirm our Interpretation of this observation. 

It is the second canonical correlation that Is of most Interest, for this has to do 
with relationships between Importance ratings and skill scores after the variance due to 
the first canonical correlation has been "extracted," that is, statistically controlled 
or taken account of. In the French group, this second canonical correlation Is .12— a 
rather small value, but nevertheless statistically significant beyond the 12 level of 
confidence. Looking at the canonical regression weights, we notice that there Is a 



116 



Table 6.10 



Inter correlations of FL Skill Importance Ratings and 
Correlations with MLA Skill Scores, By Language 

N = 2340 "Regular" Cases in Four Languages 



Importance Ratings MiA Skill Scores 





L 


s 


R 


w 


L 


s 


R 


w 








FRENCH (N = 


1114) 








Mean 


2.89 


2.91 


2.82 


2.67 


44.11 


73.55 


48.81 


47.15 


S.D. 


.32 


.30 


.42 


.54 


7.62 


9.25 


8.59 


9.45 


L 


1.00 


.51 


.14 


.25 


.14 


.18 


.10 


.14 


S 


.51 


1.00 


.05 


.31 


.08 


.16 


.05 


.08 


R 


.14 


,05 


O 

O 


.54 


.14 


.14 


.16 


.14 


W 


.25 


.31 


.54 


1.00 


.13 


.17 


.16 


.18 










GERMAN (N = 


324) 








Mean 


2.83 


2.82 


2.81 


2.56 


43.44 


86.75 


51.03 


52.09 


S.D. 


.45 


.49 


.42 


.61 


8.62 


12.80 


9.75 


13.32 


L 


1.00 


.78 


.29 


.43 


.23 


.25 


.17 


.24 


S 

1 


.78 


1.00 


.26 


.51 


.19 


.21 


.13 


.19 


1 

R 


.29 


.26 


1.00 


.48 


.26 


.20 


.26 


.27 


W 


.43 


.51 


.48 


1.00 


.21 


.23 


.39 


.23 



RUSSIAN (N = 88) 



Mean 

S.D. 


2.75 

.48 


2.77 

.45 


2.89 

.35 


2.52 

.56 


42.56 

5.99 


78.34 

12.67 


41.28 

9.64 


62.10 

10.89 


L 


1.00 


.53 


-.10 


.36 


.14 


.01 


.16 


.07 


, S 


.53 


1.00 


-.02 


.47 


.11 


.05 


.08 


.09 


R 


-.10 


-.02 


1.00 


.24 


.18 


.21 


.11 


.35 


W 


.36 


.47 


.47 


1.00 


.05 


.05 


.05 


.14 


SPANISH (N = 814) 


Mean 


2. 90 


2.94 


2,77 


2.70 


44.47 


84.01 


47.82 


54.60 


S.D. 


.33 


.27 


.45 


.51 


6.79 


12.00 


8.37 


10.11 


L 


1.00 


.46 


.18 


.13 


.11 


.08 


.08 


.11 


S 

Q 


.46 


1.00 


.15 


.19 


.04 


.02 


.00 


.05 


0 

R 


00 

. 


.16 


1.00 


.59 


.10 


.10 


.13 


.15 


W 


.13 


.19 


.59 


1.00 


.14 


.13 


.12 


.16 




i 
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^ Table 6.11 

First Two Canonical Correlations between FL SI ill Importance 
Ratings and MLA Skill Scores, and Canonical Regression Weights 



1st Canonical Correlation Ilnd Canonical Correlation 

Importance MLA Skill Importance . MLA Skill 



Ratings Scores 


Ratings 


Scores 


FRENCH (N = 1114) 




Rj « .24** 


%I = 


.12** 



Listening 


.62 




-.12 


.05 


.17 


Speaking 


.31 




.90 


.81 


.63 


Reading 


.44 




.09 


-.14 


-.70 


Writing 


.57 




.41 


-.56 


-.28 








GERMAN < 


(N = 324) 








Rj = .34** 






Rj j ® • 18 


Listening 


.71 




.43 


.22 


-.01 


Speaking 


-.04 




.32 


.39 


.66 


Reading 


.61 




-.37 


-.84 


-.75 


Writing 


.34 




.76 


.29 


.03 








RUSSIAN 


(N = 88) 








Rl » .41 






CM 

CM 

• 

II 

M 


Listening 


-.11 




-.12 


.86 


.75 


Speaking 


.16 




.01 


.13 


-.37 


Reading 


.97 




-.39 


.22 


.46 


Writing 


.15 




.91 


-.44 


-.30 








SPANISH 


(N = 814) 








Kj = .20** 






Rll = .11 


Listening 


.62 




.37 


.21 


.77 


Speaking 


-.15 




.19 


.17 


.00 


Reading 


.27 




-.25 


-.83 


-.60 


Writing 


.73 




.87 


.49 


-.22 



**p < .01 
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palring of signs such that in making the best prediction of skill scores from importance 
ratings after the first canonical variance has been taken account of, we would weight 
both importance ratings and skills scores in Listening and Speaking positively » and both 
Importance ratings and skills scores in Reading and Writing negatively * In other words, 
the difference between Importance ratings for oral and written skills predicts a corres- 
ponding difference between test scores. This is cleat evidence that at least in the 
French group, students tend to get higher scores in skills to which they attach greater 
importance. The causal relationship, however, is not specified! it may be that student 
attitudes cause greater devotion to the attainment of specific skills, or it may be that 
greater success in particular skills~for whatever reasons—causes the student to attach 
greater importance to those skills. Or the causal relationship may go both directions 
for different groups of students. Ail we can say is that the results show a definite 
tendency for student ratings of importance for different skills to be related to their 
differential degree of success in those skills. 

Results for the other languages are not so clear as they are for French, but there 
are trends of the same general type. For example, in the German group the difference 
between Importance ratings of Speaking and Reading predicts the relative degi.ve of meas- 
ured attainment in those skills. In Spanish the difference between imp’ortance ratings 
of (in particular) Writing and Reading predicts the difference between relative attain- 
ments in (in particular) Listening and Reading; that is, the student who is more oriented 
toward Writing (and also Listening and Speaking, to some extent) is better in oral skills 
than in written skills. One could give an ad hoc interpretation of this, but it is 
probably useless to do so in view of the statistical non-significance of the findings. 

Relations of Vocational Plans and Other Interests to Foreign L anguage Skill 
Attainment 

For 2340 "regular cases" in four languages, means and standard deviations of MLA 
skills test scores were computed for groups indicating various vocational plans, and com- 
pared statistically with the data for each language group as a whole. The main results 
are shovm in Table 6.12. A remark shout the constitution of the groups and the statis- 
tical computations is in order. Daf a are shown in the first row for "all planning to 
teach," i.e., all who marked the blank in question 1 on page 4 of the Student 

Questionnaire (Appendix B) . In the statistical computations, the scores of this group, 
for each language, were compared with those of all who did niT»t mark this blank (or any of 
the Indented spaces immediately under it). For reference, the data for the total group 
in each language are shown in the last row of the table. Immediately under the row 
(actually, two rows, one for means, one for standard deviations) for all planning to 
teach" are indications as to the statistical significance of the comparisons; the footnote 
of the table shows the interpretation of these indications. In the next row, data are 
shown for those planning to teach "at the elementary school level"; this is of course a 
subgroup of all planning to teach. Nevertheless, statistical comparisons for this group 
are made not with those planning to teach at other levels but with all who did not indi- 
cate that they were planning to teach at the elementary level; e.g., for French, the 111 
cases planning to teach at the elementary level were compared with the remaining (1114 
— Ill) * 1003 cases. Similar procedures* necessitated by the fact that there were many 
multiple responses, were used throughout the table. The reader who wishes to make dif- 
ferent types of statistical comparisons may do so by using the N*s, means, and S.D. s 
given, but he must do so with caution because in many cases the N*s overlap to some 
extent. For example (as may be seen in Table 6.9), there were many students who marked 
several levels at which they planned to teach. It should be noted, of course, that the 
degree of statistical slsnificance found is partly a function of the sizes of the respec- 
tive samples, and therefore it is legitimate to examine the absolute sizes of means in 

order to gain an idea of trends. 

The main points of interest in Table 6.12 are as follows. 

(1) Those planning to teach at some level, as compared to those who do not plan to 
teach at any level, are seldom significantly different, but when they are, they are 
slightly superior. 



Means and S.D. *s of MLA Skills Test Scores by Vocational Plans (Sexes Combined), and by Major Language 
2340 "Regular" Cases In Four Languages 
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significantly (p < .01) superior to remainder of total group 
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(2) Those planning to teach at the elementary school level are generally represent- 
ative of the total group. (The N’s, of course, are quite small.) 

(3) Those planning to teach at the secondary school level tend to be significantly 
inferior to thcL withLt such plans. This is particularly true for the French and 
Spanish groups. Nevertheless, the absolute differences are not large. 

(4) Those planning to teach at the college level are strikingly superior to the 
reaalnder of the group. This is true in all languages. In fact, the superiority of the 

groups oriented tl college teaching is the only really stand-out ^ 

u rtno fKflv talc** some comfort in this fact, for the sake of the colleges and 

iSiiCTslties, but on’^the other hand the lower levels of skill exhibited by th<^ remark^ 
to teach at elementary and secondary school levels leave much to be f ^ 

in Chapter IV, the overall skill levels of the CFLT sa^le 1“ ^AinrP«fi“- 

eenerallv at the S-2 or S-2+ levels on the FSI scale, i.e., at Limited working rroric 
ency" o/a little higher. Even the college-teaching-oriented groups are not much better 
on this scale, on the average; only in German and in Spmish do these groups generally 
attain S-3, "Minimum Professional Proficiency, on the Fol scale. 

(5) Students who indicated that they planned to use FL skills in some kind of future 

emplo^Lt other than teaching tend to be slightly ‘d 

maindM of the group. The differences are generally significant only in rte ^ 

erauos hovrever. and In any case the differences are not large. One may inte^ret 
thls^resSlt to ’mean that students who have definite vocational goals other than 
tend to take their foreign language training more seriously than students who do not have 

such goals. 

Students oriented to careers in some kind of government work (foreign service, 
united Nations! etc.) , or in translating «.d interpreting, are somewhat super- 

±oT j in general, to students otherwise oriented. 

(7) Students OTi.ented to careers in business or in linguistics are generally 
sentative of the total group. The finding Tor students thinking of careers in linguist 
is somLhat counter to expertation; perhap' the students concerned do not have a good idea 

of what linguistics is. 

Data were also tabulated for students who marked reasons other than vocational as 
promp^ng Ih» ?o choose a foreign language major. A great m»y of these students had 
Slso marked vocational reasons (teaching and other future 

since they were asked to check as many reasons as applied. The results are sn^ 

Table 6.13. Because of multiple responses, the same type , ._„i- 

Med arin Table 6,12. For reference, the first row give, the data for the total smpU, 
Sentical to those shown in the last row of Table 6.12. The major observations to be 

nftdft from Table 6 <>13 are as follows* 

(1) None of the groups indicating any of the reasons for choosing a ««®lS“ 

but they do not appear to be of much prac^:ic^ significance. 

Over all languages, statistically significant differences are most likely to ap- 
pear for groups indicating that an important reason for choosing a 

w« tia^ Sey were "Interlsted in Intercultural f 

groups generf^lli had slightly higher than average scores on the MLA skills tests. 

Slight superiority tended also to occur for groups indicating that they expected 

to use^ih:ir‘'?or:rg:'i»gulEe skills in rr^l or srudy abro^. and or 

they majored in a foreign language primarily because of interest in its literary 

cultural aspects . 



Table 6.13 

Test Scores, by Reasons Other than Vocations for Choosing a 
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(4) Groups indicating that they majored in a foreign language "mainly for interest 
and enjoyment." or simply because they "needed a major," were representative of the total 
group. One might perhaps have expected that these groups would be slightly Inferior to 
the total group in foreign language skills, but such was not the case. 

5, Major Dimensions of a "Typology" o f FL Majors 

This section presents an attempt to use advanced statistical techniques to develop a 
"tvpology” of foreign language majors. Essentially, the technique involves finding clus- 
ters of questionnaire items that are answered in more or less the same pattern, either in 
one direction or its opposite. Because the procedure typically involves applying factor 
analysis to questionnaire items, it has been called the "QFA" technique. It has been 
described by Carroll et , (1966, Appendix C) , and its application to the present Preb- 
le was developed by Fannie. A. Handrick. 

For the national sample, the technique was applied to selected questionnaire respon- 
ses of 2571 students. These were all the students in the sample (including students in 
all five languages) who answered all questionnaire items involved in the analysis. Both 
"regular" and "odd" cases were included (see Chapter III, p.39). 

The questionnaire items selected for study were item 5 on page 1, and items 1-11 on 
pace 4. of the Questionnaire for Foreign Language Majors . Responses to item 5 were re- 
coded so that they could be interpreted as "yes" or "no" answers to the question, 'Have 
you studied any foreign language other than your major language? 

Technical note. As a binary item with two possible answers this item contri- 
buted, in effect, two variables to the QFA covariance matrix. Each of the 18 res- 
ponse positions in item 1, page 4, was regarded as a binary item also; these 18 
positions contributed therefore 36 variables to the QFA covariance matrix. Each of 
the 35 remaining response positions on page 4 (items 2 to 11) was regarded as a 
position on a scale and therefore contributed one variable to the QFA matrix, except 
that an extra "dummy” response position was created to represent non-response to 
item 8 (tantamount to having responded yes to either question 6 or 7) . Thus, the 
QFA matrix contained 74 variables in all. The complete covariance matrix, generated 
by considering each variable as containing l*s or 0*s depending upon whether that 
variable was coded, was submitted to a principal axes factor analysis computation 
routine. It yielded two latent roots greater than unity, and 43 positive latent 
roots in all; the remaining 31 roots were zero, corresponding to the statistical re- 
dundancy introduced by the use of non-independent responses. (For exa^le, a s u en 
who marked "yes" in a two-choice answer would obviously not also mark no. ) Al- 
though only two latent roots were greater than unity, at least 6 of the dimensions 
appeared to be interpretable after orthogonal rotation. 

Table 6.14 shows factor loadings for selected variables on axes rotated orthogonally 
from the principal axis factors. Redundant variables, and variables with low communal- 

itles, were excluded from the table. 

The six factors found represent independent ways in which the FL majors in the sample 
varied. 

Probably the most important way in which they vary is whether they plan to teach, or 
plan to use their foreign language skills in business or professional work. This dimen- 
sion is reflected in Factor A, whose larger factor loadings are as follows: 

Factor A loading 

Plans use of FL in business or professional work *61 

Plans use in government work 

20 

Plans use in business 

Plans use as a translator or interpreter 18 
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Table 6.14 

Factor Loadings for Selected Questionnaire Responses 
(Orthogonal Rotated Axes) 



Factor 



Variable 


A 


B 


C 


D 


E 


F 


Has had other mod. FL's 


.03 


.02 


-.03 


.04 


.05 


.12 


Flans to teach at some level 


-.18 


.05 


.34 


.08 


.02 


.04 


Flans to teach at sec. sch. level 


-.21 


.09 


.40 


-.07 


.00 


.04 


Flans to get teaching license 


-.22 


.03 


.37 


-.07 


-.01 


.01 


Flans to teach, college level 


-.01 


.05 


.01 


.24 


.05 


.00 


Flans use In business or prof, work 


.41 


-.02 


.02 


.00 


.11 


.03 


” ” " business 


.20 


-.02 


-.01 


-.02 


.05 


.00 


” " " government work 


.29 


-.02 


.02 


-.01 


.08 


.03 


" ” as translator or Interpreter 


.18 


.00 


.03 


.03 


.09 


.02 


Flans use In travel or study 


.06 


.04 


.08 


-.10 


.26 


-.03 


Majored for Interest and enjoyment 


-.12 


.01 


-.09 


-.24 


.22 


.02 


Interested In lit. and civilization 


-.13 


-.02 


-.09 


.11 


.34 


-.01 


Interested In International understanding 


.02 


.05 


.03 


-.05 


.34 


-.04 


Listening Is of great Importance 


.02 


.04 


.07 


.03 


.07 


-.03 


Reading " " " 


-.06 


.01 


.00 


.17 


.08 


.02 


Writing " " 


-.06 


.04 


.07 


.23 


.09 


-.03 


Has taken Informal courses 


.02 


.24 


-.05 


.01 


.06 


.00 


Took Latin or Greek In high school 


.00 


.05 


-.05 


.00 


.02 


.45 


Had summer or school year abroad 


.06 


.43 


-.06 


-.03 


.01 


-.01 


No use of FL at home 


-.01 


-.03 


-.03 


-.12 


.00 


.09 


Moderate outside use of the FL 


.01 


-.01 


.02 


-.03 


.10 


-.01 


Has read at least 3 books In FL 


.06 


.16 


-.05 


.22 


.09 


-.04 
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Plans to teach at some level. ••••••••••••••• -bIo 

Plans to teach at secondary school ••••••••••• • -.21 

Plans to get a teaching license -•22 

The negative loadings reflect the fact that those who state that they plan to use 
their foreign language skills In business or professional work show a considerable ten- 
dency not to Indicate any plan to teach. 

Factors C and D Indicate that planning to teach at the secondary school level Is 
largely Independent of planning to teach at the college level; that la, the amount of 
overlap between those planning to teach at the secondary school and those planning to 
teach at the college level Is only about what would be expected by chance. Planning to 
teach at the college level Is associated with several other variables loaded on Factor D. 



Factor C loading 



Plans to teach at the secondary school level .40 

Plans to get a teaching certificate 37 

Plans to teach (at some level) 34 



Factor D loading 

Plans to teach at the college level ............ *24 

Writing skill Is thought to be of great Importance 23 



Has read at least 3 books In the 22 

Reading skill Is thought to be of great Importance ..... .17 

Majored primarily for Interest and enjoyment ........ -.24 



The negative loading In the case of the last variable shows that those who plan to 
teach at the college level have a considerable tendency not to Indicate that they choose 
a FL major primarily for "Interest . and enjoyment"; that Is, their plans for college teach- 
ing are of a serious nature. 

Factor B Is a dimension that differentiates those who have studied abroad from those 
who have not; the loadings also Indicate that those who have studied abroad are quite 
likely to have taken Informal courses In their major language and to have done considerable 
reading. 



Has had a summer or year of study abroad 
Has taken Informal courses In the FL . . 
Has read at least 3 books In the FL . . 



Factor B loading 
. .43 

. .24 

. . .16 



Factor E seems to be an "Interest" factor, differentiating those who are attracted 
by the Intrinsic benefits of foreign language study from those who do not seem to eare 
about these things: 

Factor E loading 



Interested In the literature and civilization of the 

foreign country *34 

Interested In International understanding and communication. .34 

Plans use of the FL In travel or study abroad 26 

Majored primarily for interest and enjoyment 22 






Finally, Factor F seems to represent simply the dimension of whether the student 
has studied Latin, Greek, and other languages besides his major language: 

Factor F loading 

Took Latin or Greek in high school 

Has had other modem foreign languages besides his 
major language 
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Chapter ¥11 



EXPERIENCE AND TRAINING FACTORS IN FOREIGN LANGUAGE SKILL ATTAINMENT 



1. Introduction 

In this chapter we report the results of analyses in which the relations between 
tested foreign language attainment and various experiential and training variables were 
explored. Many of these variables represent kinds of Influences to which the students 
had been exposed, whether by their own choice or by the force of circumstances. It will 
be tempting to infer, whenever a strong relationship between one of these variables and 
measured foreign language skill appears, that the "influence" in question was a causal 
one. Often such an inference will be very strong, particularly when it is supported by 
a network of related findings all pointing in the same direction. It must be remembered, 
however, that this study was of a purely descriptive nature and can only claim to dis- 
cover interesting and suggestive relationships. The causal basis of these relationships 
could only be Inverstigated properly by studies in which the variables are actually mani- 
pulated over time. For example, one of the variables to be discussed below is the ex- 
tent to which the student has had the opportunity to go abroad, either as a student or 
as a tourist, to the country where his major foreign language is spoken. We find that 
students who have been abroad are better in foreign language skills than those who have 
not been abroad. It is perhaps reasonable to infer, from common sense and from common 
observation, that experience in foreign countries actually has a causal influence on the 
improvement of foreign language skills. On the other hand, we do not know anything about 
the foreign language attainments of the students before they went abroad; it is conceiv- 
able that students who went abroad were the better language students before they went 
abroad, and that the experience abroad really had little or nothing to do with their 
attainments. As a matter of fact, many institutions follow the practice of selecting 
only their better students for admission to programs of study abroad. Also, the prospect 
of even a short trip abroad may stimulate the student to improve his skills markedly even 
before he gets on the plane or ship. Therefore, from the finding that there is a relat- 
ionship between measured foreign language skill and whether the student has had a tour 
abroad it does not inevitably follow that experience abroad causes his language skills 
to increase. The descriptive findings in this chapter must be taken as merely suggestive 
of possible causal relationships. If the language used in this chapter occasionally 
seems to imply causal interpretations, it is only because it io sometimes difficult to 
find words which are sufficiently neutral in this respect. 

The design of this chapter is to consider the Svsveral experiential and training 
factors one by one, or at most in pairs. Chapter Vlll will present more elaborate stat- 
istical analyses in which large numbers of variables are considered together. 



2. Time of Beginning Foreign Language Study 



Our sample was composed of individuals graduating in the class of 1965. On the 
assumption that the typical individual went through a normal course of schooling without 
skipping or repeating grades, he would have entered the first grade in 1949, junior high 
school (Grade 7) in 1955, senior high school (Grade 10) in 1958, and college in 1961. 

A number of public schools, e.g. those of Cleveland, Ohio, as well as many private 
schools, have had foreign language programs in elementary schools for several decades, 
and it is possible that some of the sample had been to such schools in the early grades. 
Around 1953, when our sample was typically in grade 5, the "FLES" (Foreign Language in 
Elementary School) movement, whereby increasing numbers of schools instituted foreign 
language programs in pre-junlor-hlgh grades, had started to gain momentum, and it is not 
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unreasonable to expect that many of the sample had had the opportunity to start foreign 
language study at that time. 

On pages 2 and 3 of the Questionnaire for Foreign Language Majors* the students 
were asked to list all courses in their major foreign language they had taken from grade 
school up through the time of testing. On the basis of this list* they were classified 
in terms of whether they began the study of their foreign language in "grade school*" in 
high school* or in college. (Students who listed no courses at the grade school or high 
school level and no beginning language course at the college level were relegated to an 
"odd type" category and were eixluded from further analysis.) Table 7.1 shows the num- 
bers and percentages of each category in each language* for all "regular" cases in all 
languages except Italian. 



Table 7.1 

Numbers and Percentages of "Regular" Cases* 
by Time of Beginning FL Study* by Language (Excluding Italian) 
(All cases complete on Four MLA Skills Tests) 











LANGUAGE 












Time of Beginning 


French 
N % 


German 
N % 


Russian 
N % 


Spanish 
N % 


Total 
N % 


"Grade school" 


148 


13.3 


7 


2.1 


2 


2.2 


96 


11.7 


253 


10.8 


High school 


785 


70.4 


147 


44.0 


22 


24.7 


557 


60.3 


1511 


64.4 


Colleger 


182 


16.3 


170 


50.9 


65 


73.1 


163 


20.0 


580 


24.8 


Total 


1115 


100.0 


324 


100.0 


89 


100.0 


816 


100. 0 


2344 


100.0 



There are no surprises in this table* except possibly the rather substantial numbers 
of cases who reported courses at the grade school level in French and Spanish: 13. 3Z in 
French and 11.7% in Spanish. Most students of French and Spanish* however* started at 
the high school level* and a slight majority of students in German and Russian started 
at the college level. 

Next* the mean MLA skills test scores for the different groups were computed. For 
this analysis* grade school and high school starters were combined in the case of German 
and Russian because of the small numbers of students starting in the elementary school. 
The results are shown in Table 7.2. The differences were tested for statistical signif- 
icance by one-way analysis of variance. 

In French and Spanish* the two languages which substantial numbers of students 
started in grade school* there are highly significant differences amos'ig the categories. 

Ill all tests* those who started at grade school were superior to those who started in 
high school* and these in turn were better than those who started in college. In French* 
the successive differences are approximately equal; that is* the differences between 
grade school starters and high school starters and between high school starters and col- 
lege starters are similar. In Spanish* the spread between the grade school and high 
school starters is wider than that between the high school and college starters. 

The trend is in the same direction in Russian* although the differences show up with 
less statistical significance because of the small numbers of cases* and of course the 
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Table 7.2 

Means and S.D.*s of MLA Skills Test Scores for Groups Starting 
FL Study at Different Educational Levels 
(All "Regular” Cases Complete on MLA Skills Test Scores) 



Tine of Listening Speaking Reading Writing 



Beginning 


N 


X 


a 


X 


a 


X 


0 


X 


a 








FRENCH 












Grade School 


148 


47.47 


6.46 


78.09 


8.50 


51.64 


7.75 


51.40 


8.61 


High School 


785 


44.21 


7.54 


73.48 


8.95 


48.85 


8.51 


47.32 


9.36 


College 


182 


40.99 


7.63 


70.19 


9.58 


46.30 


8.80 


42.99 


9.35 


Total 


1115 


44.11 


7.62 


73.55 


9.25 


48.81 


8.59 


47.15 


9.54 


F 




31. 


2** 


31. 


5** 


16 


.2** 


33.9** 








GERMAN 












Gr.Sch.& H.S. 


154 


42.80 


8.95 


86.53 


13.64 


50.64 


10.00 


51.14 


13.76 


College 


170 


44.02 


8.26 


86.^5 


11.99 


51.38 


9.51 


52.94 


12.86 


Total 


324 


43.44 


8.62 


86.75 


12.80 


51.03 


9.75 


52.09 


13.32 


F 




0.( 




0.] 




0. 


5 


1.5 










RUSSIAN 












6r.Sch.& H.S. 


24 


44.00 


5.73 


78.79 


9.83 


44.63 


9.06 


64.42 


10.32 


College 


65 


41.98 


5.96 


78.11 


13.48 


40,00 


9.48 


61.20 


10.89 


Total 


89 


42.56 


5.99 


78.34 


12.67 


41.28 


9.64 


62.10 


10.89 


F 




2. 


0* 


0 


.0 


4, 


.2** 


1. 


5 








SPANISH 












Grade School 


96 


48.60 


5.31 


89.43 


13.91 


52.83 


7.71 


60.18 


9.02 


High School 


557 


44.29 


6.76 


83.60 


11.31 


47.44 


8.43 


54.32 


10.04 


College 


163 


42.66 


6.65 


82.15 


12.23 


46.21 


7.58 


52.37 


9.84 


Total 


816 


44.47 


6.79 


84.01 


12.00 


47.82 


8.37 


54.60 


10.11 


F 




25. 


2** 


12 


.4** 


21. 


.7** 


19. 


6** 





** p <.01 

* p <.05 
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grade school and high school starters were combined. 

The trends are reversed in German, but the differences in means nowhere reach stat- 
istical significance despite the sizable^ numbers of cases. Again, the small number of 
grade school starters was combined in the analysis with high schoo]. starters. 

No immediate explanation of these discrepant trends in German is available. 

The clearly significant treads in French and Spanish, with the parallel trend in 
Russian, call for careful examination. The simplest qnd most conservative explanation 
of the trend is that the attainment of skill in a foreign language is a function of the 
amount of time spent in its study; it may be assumed that most of the students involved 
in the investigation had studied the foreign language more or less continuously since 
the time of first starting that study. It is not surprising, therefore, that those who 
started earliest had, on the average, achieved the most by the time of college graduation. 

Some readers may be tempted to Interpret these findings as telling justification for 
the FLES movement. Such a conclusion would be premature. It would be justification for 
the teaching of foreign languages in the elementary school only if it could also be 
shown that it is critical or necessary that the start of language study must be in the 
grade school. This could hardly be the case, for many students found in our sample who 
started in high school or college did as well as many students who started earlier. It 
must be pointed out, also, that the small number of people who appear In our data as 
having started their foreign language in grade school probably constitute only a small 
fraction of all those who started foreign language in grade school at the same time. 

Any conclusion about the value of having started a foreign language in the elementary 
school would have to take account of what happened to the rest of the students in the age 
cohort who started a foreign language in grade school. Our data therefore cannot be used 
to give unqualified support to the FLES movement. The conclusion that does seem to 
emerge from the data is that for those students who were enabled to start French or Span- 
ish in the elementary school and who liked the language well enough to impel them to con- 
tinue with it to the point of graduating from college with a language major, their start 
in elementary school gave them a distinct advantage, on the average, over those who 
started later. 

In any case, the data presented thus far merely show the overall differences among 
those who started at various educational levels. In order to make stronger Inferences 
about the effects of having started early, we must rule out other possible sources of 
the differences observed, such as differences in language aptitude or in other kinds of 
educational advantages. Such possibilities will be discussed at a later point in this 
report. 

It might be asked whether Time of Beginning is in any way related to language apti- 
tude. On the one hand, there might be selective factors whereby students beginning 
language study early (and continuing) might be students with greater language aptitude 
than those starting later. On the other hand, early starting might have the effect of 
enhancing language aptitude in some manner that would be reflected in language aptitude 
test scores. The data available in this investigation cannot give reliable information 
that would point definitely to one rather than the other of these possibilities; ques- 
tions concerning explanation of any relationships found between language aptitude and 
Time of Beginning FL Study could be answered only by studies specifically designed to 
answer these questions. Nevertheless, the data assembled here may be of some interest; 
they are shown in Table 7.3. The differences in language aptitude among the various 
starting groups are in every case quite small in absolute magnitude. Only some of them 
are statistically significant, and they do not form a consistent pattern. For example, 
in French, early starters are significantly superior on MLAT-3, a test of "phonetic 
coding ability," but are significantly inferior on MLAT-4, a test of "grammatical sensi- 
tivity," and show no significant trend on MLAT-3. Nevertheless, the slightly inferior 
performance of college starters on MLAT-3 holds up across languages— as a significant 
effect in the Spanish group, and as a non-significant trend in the German and Russian 
groups. 
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Table 7.3 

Mean MLAT Scores for Groups Starting their Major 
Foreign Langijiage at Different Tiriies 



Time of beginning study 
of major language 


N 


Mean Scores i 


on Sections of 


the MLAT 


Spelling Clues Words 
MLAT-3 


in Sentences 
MLAT-4 


Paired-Associates 

MLAT-5 


< 




FRENCH MAJORS 






Grade school 


146 


28.45 


30.36 


18.99 


High school 


784 


27.62 


32.48 


19.48 


College 


179 


25.75 


32.26 


19.65 


Total 


1109 


27.43 


32.17 


19.69 


F-ratio 




3.6* 


6.8** 


1.8 






GERMAN MAJORS 






Grade school 


7 1 










\ 


25.59 


32.33 


19.49 


High school 


131 j 


1 






College 


151 


24.87 


32.88 


19.68 


Total 


289 


25.21 


32.62 


19.58 


F-ratio 




0.4 


0.5 


0.1 






RUSSIAI^ MAJORS 






Grade school 


2*] 










I 


r 31.00 


35.00 


11.00 


High school 


20 ] 


( 






College 




28.92 


30.92 


19.97 ' 


Total 


81 


29.48 


32.02 


20.25 


F-ratio 




0.5 


4.1* 


0.7 






SPANISH MAJORS 






Grade school 


99 


27.95 


29c09 


18.83 


High school 


544 


24.82 


30.42 


19.06 


College 


163 


23.05 


30.08 


17.94 


Total 


806 


24.84 


30.19 


18.80 


F-ratio 




7.7** 


1.4 


2.6 



**p < ,01; *p < ,05 
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On the whole, the small magnitude of the MLAT differences across groups starting at 
different times appears to signify that MLAT scores are largely independent of time of 
starting the major foreign language. In any case, it should be remembered that groups, 
starting their major foreign language late are not necessarily without early language, 
training experiences in other languages beslues the one in which they are majoring. 



3. Instructional variables 

’« 

Besides data on the time of beginning FL study, considerable information was avail- 
able as to the numbers and types of courses the students had had, as well as the specific 
kinds of instructional experiences they had had. 

It was possible, first of all, to analyze more in detail the numbers and kinds of 
courses that students in different starting groups had had at three educational levels. 
Table 7.4 shows these data along with mean MLA skills test scores. What is clear from 
the table is that the achievement of the students on the tests can be interpreted as as- 
sociated with the amount of time they had spent on language study. The substantial 
numbers of students who started French or Spanish in elementary school were likely to 
take more courses in high school; by the time they arrived in college they evidently did 
not have to take as many purely "language" courses as those who started in i.igh school or 
college, and they tended therefore to take more foreign language literature courses. As 
a result, we may presume, they were able to achieve higher scores on the skills tests. 
(The data for the small numbers of elementary school starters in German and Russian are 
generally consistent with the findings for french and Spanish.) Likewise, those who 
started in high school had to take fewer college language courses and were enabled to take 
more literature courses than those who started in college; their MLA test scores were cor- 
respondingly higher. This latter statement is true, at least, for students in French and 
Spanish; the smaller number of college literature courses taken by high school starters in 
German and Russian may have been restricted by the limited availability of advanced col- 
lege literature courses in those languages. 

It would be desirable to define as a variable the total number of semesters the 
student had studied a foreign language, but it was not possible to do so from the data, 
not only because data were unavailable but also because it would be impossible a priori 
to equate semesters or hours at the various levels. (Students reported number of credit 
hours at the college level, but the number of credit hours per course varied widely.) 
Traditionally, foreign language teachers have considered one year of a hl^h school course 
to be equivalent to one semester of a college course. If we accept that assumption, and 
assume further that three semesters of work at the grade school level is equivalent to 
one semester at the college level, and that the typical college semester ha& three credit 
hours, we can compute the total number of "virtual college semesters" each starting group 
had; these data are shown in one of the columns of Table 7.4. The rank orders of the 
resulting figures, across starting groups, correspond closely to those of the MLA skills 
test means. It would appear that perhaps the reason why those who started German in 
college did slightly better on the tests than those who started in high school is that 
they had slightly more total exposure to the language in terms of "virtual college sem- 
esters." It is also noteworthy that the total exposure figures are greatest for the 
French group as a whole and least for the Russian group as a whole; this may explain the 
fact that the Russian group did relatively poorly on the tests when evaluated against FSI 
equivalents . 

There exists a somewhat better procedure for showing the relative values of work at 
different educational levels, namely, linear regression analysis. Since this technique 
necessitates fairly largn numbers of cases to show reliable findings, it was applied only 
for the French and Spanish groups, with results shown in Tables 7.5 and 7.6. This tech- 
nique determines the optimal linear weights by which the predictor variables may be multi- 
plied to yield a prediction of a dependent variable. Thus, if we examine the "b-welghts" 
in the table for the French group (Table 7.5), we can multiply the number of semesters in 
grade school by .3180, the number of semesters In high school by .4698, the number of 
hours of college language by .0797, and the number of hours of college literature by 
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Table 7.4 



Mean Semesters or Course Hours Taken by Groups 
Starting at Different Times » with Mean MLA Skills Test Scores 






Group 



Sem. 

Grade Sem. 
N School H.S. 



Krs • Hrs • 
College College 
"Grammar" Lit • 
Courses Courses 



Total 

"Virtual 

College 

Semes- 

ters" L 



MLA Means 

”"S R W 



Began in grade sch. 


139 


4.99 


FRENCH 

6.63 13.37 


20.84 


16c38 


47.3 


77.9 


51.4 


51.2 


Began high school 


733 


0.00 


5.29 


17.58 


18.28 


14.60 


44.2 


73.5 


48.9 


47.3 


Began college 
Total 


167 


0.00 


0.00 


26.85 


14.01 


13.62 


41.2 


70.1 


46.6 


43.3 


1039 


0.67 


4.62 


18.50 


17.94 


14.68 


44.1 


73o5 


48.9 


47.2 


Began In grade sch. 


7 


10.86 


GERMAN 
6.57 10.00 


18.14 


16.28 


55.7 


105.3 


62.7 


66.1 


Began high school 


131 


0.00 


4.38 


15.19 


16.95 


12.90 


42.0 


85.8 


50.1 


50.5 


Began college 


151 


0.00 


0.00 


22.86 


17.12 


13.32 


43.7 


86.3 


51.0 




Total 


289 


0.26 


2.14 


19.07 


17.07 


13.20 


43.2 


86.5 


50.9 


51.9 


Began in grade sch. 


. 2 


1.60 


RUSSIAN 
3.50 6.41 


15.44 


9.57 


42.4 


72.7 


45.5 


66.5 


Began high school 


20 


0.00 


3.25 


23.53 


9.81 


12.74 


44.4 


79.3 


45.6 


65 .0 


Began college 


58 


0.00 


0.00 


26.81 


10.14 


12.32 


42.4 


78.7 


40.5 


61.5 


Total 


80 


0.04 


0.90 


25.48 


10.19 


12.35 


42.9 


78.7 


41.9 


62 . 5 


Began in grade sch 


. 91 


4.46 


SPANISH 
5.01 13.16 


19.49 


14.87 


48.8 


89.9 


53.0 


60.1 


Began high school 


518 


0.00 


5.15 


15 .06 


17.74 


13.51 


44.3 


83.5 


47.5 


54.3 


Began college 


154 


0.00 


0.00 


25.17 


14.84 


13.34 




81.9 


46 .3 


52.2 


Total 


763 


0.53 


4.10 


16.88 


17.36 


13.64 


44.5 


84.0 


47.9 


54.6 





riiilflih 
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Table 7.5 



HegtessKm System for Predicting MIA Skills Test Scores 
from Amount of Study at Different Educational Levels 

FRENCH 



A. Basic Data for Total Group <N - 1039) 

Predictor Variables 



Criterion Variables (MLA Tests) 

Read. Write. 



Mean: 
S sD. : 



No. sem. grade sch. 

No.sem. high sch. 

Hrs. college 
lang. courses 

Hrs. college 
lit. courses 



1 

2 

3 

4 



L 

S 

R 

W 



^ 

0.67 


4.62 


18.50 


17.94 


44.14 


73.52 


48.89 


47.18 


2.42 


2.84 


9.99 


8.04 


7.59 


9.67 


8.51 


9.53 


In ter cor re lations 






Beta-weights 


.1002 


1.00 


.21 


-.17 


.13 


1 .1015 


.1262 


.0847 


.21 


1.00 


-.42 


.32 


2 .1758 


.1984 


.0954 


.1484 


-.17 


-.42 


1.00 


- 21 


3 .1046 


.1217 


.0158 


.0354 


.13 


.32 


-.21 


1.00 


4 .3025 


.1756 


.2918 


.2669 








Multiple 

R: .4010 


.3241 


.3544 


.3654 


Correlations 


with Criteria 




b~welghts 




.16 


.25 


-.05 


.35 


1 .3180 


.4830 


.2978 


.3945 


.17 


.23 


CM 

O 

• 

1 


.23 


2 .4698 


.6480 


.2859 


.4984 


.14 


.20 


-.10 


.33 


3 .0797 


.1131 


.0136 


.0334 


.16 


.24 


-.10 


.32 


4 .2854 


.2021 


.3089 


.3164 






Intercept : 


35.1620 


64.4846 


41.5764 


38.3190 



B. Prediction of Criterion Means for Groups Starting at Different Times 

Predicted Values of Criterion 



Began in Grade School 
Began in High School 



139 



733 



Means 

(Actual 


Values in Parentheses) 


46.88 


76.91 


51.58 


50.63 


(47.34) 


(77.88) 


(51.45) 


(51.23) 


44.27 


73.60 


48.97 


47.33 


(44.20) 


(73.47) 


(48.93) 


(47.31) 


41.30 


70.35 


46.27 


43.65 


(41.22) 


(70.14) 


(46.59) 


(43.26) 



Began in College 



167 
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Table 7.6 



Regression System for Predicting MLA Skills Test Scores 
from Amount of Study at Different Educational Levels 

SPANISH 

A. Basic Data for Total Group (N “ 763) 

Predictor Variables Criterion Variables (MLA Tests) 





1 


2 


3 


4 




Listen. 


Speak. 


Read. 


Write 


Mean: 


0.53 


4.10 


16.38 


17.36 




44.50 


83.95 


47,88 


54,59 


S.D.: 


1.91 


2.78 


9.39 


7.94 




6.32 


12.07 


8.39 


10.19 




Intercorrelatlcns 






Beta-weights 




No. sem. grade sch. 1 


1.00 


.01 


-.12 


.05 


1 


.2115 


.2509 


.1936 


.1712 


No.sem. high sch. 2 


.01 


1.00 


-.43 


..21 


2 


.0860 


.0111 


.0055 


.0853 


Hrs . . college 

lang. courses 3 


-.12 


-.43 


1.00 


-.21 


3 


-.0277 


-.0153 


-.1085 


-.0348 


Hrs. college 

lit. courses 4 


.05 


.21 


-.21 


1.00 


4 


.2856 


.1419 


.2664 


.2763 










Multiple 

R: .3974 


.2995 


.3791 


.3702 




Correlations 


with Criteria 






b-welghts 




L 


.23 


.16 


-.15 


.32 


1 


.7550 


1.5860 


.8507 


.9130 


S 


.26 


.05 


-.08 


.16 


2 


.2107 


.0483 


.0168 


.3128 


R 


.22 


.11 


-.19 


.30 


3 


-.0205 


-.0193 


-.0973 


-.0377 


W 


.19 


.16 


-.15 


.31 


4 


.2455 


.2160 


2.810 


.3546 










Intercept : 


39.3201 


79.4874 


44.1245 


47.3042 



B. Prediction of Criterion Means for Groups Starting at Different Times 



Group N_ 

Began in Grade School 91 

Began in High School 518 



Predicted Values of Criterion 
Means 

(Actual Values in Parentheses) 



L 


S 


R 


U 


48.26 


90.76 


52.20 


59.36 


(48.79) 


(89.86) 


(52.95) 


(60.08) 


44.45 


83.28 


47.73 


54.60 


(44.31) 


(83.52) 


(47.46) 


(54.32) 


42.45 


82,21 


45.85 


51.62 


(42.62) 


(81.89) 


(46.30) 


(52.23) 



Began in College 



154 
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.2854, then add 35.1620, to yield the estimated MLA Listening score for a given student. 
(These weights are to be contrasted with weights of .3333, .5000, .3333, and .3333, re- 
spectively, implied by the weighting system described above whereby the number of vir- 
tual college semesters" is computed.) It would appear that for the total French group, 
hours of college l anguage courses are of much less significance than is traditionally 
thought: they are'of relatively little weight , certainly, for those who have already 
had a solid introduction to the language in grade school or high school. On the other 
hand, college literature courses are shown to be influential in promoting language com- 
petence . 



Tables 7.5 and 7.6 show, in general, that a considerable proportion of the variance 
of MLA skills test scores is associated with the amount of time students have spent in 
language study. The mean test scores of groups starting at different educational levels 
are quite accurately predictable. Listening test scores are on the whole more predict- 
able in this way than scores on the other tests, a finding that suggests that the Listen- 
ing tests are the most valid measures of language competence. 



The negative weights of college language courses for the Spanish group suggest that 
students arriving at college with previous study who still had to take college language 
courses may have been the poorer students. (These courses would probably not have nega- 
tive weights, however, if data for students starting in college had been analyzed 
separately. ) 



Instructional Procedures 

The course Information collected from the students touched various aspects of in- 
structional procedures. We have already seen, in Chapter V, that institutions differ 
markedly in some of these variables when the institutions are classified by size and 
type (r>rlvate vs. public). Information was available on instructional procedures not only 
in college but also in high school. Insofar as the* students could be trusted to remember 
those aspects of their high school courses. The data are sufficiently consistent over 
languages, however, to justify considerable confidence in them. 



We shall focus attention on just four of these variables: 

(1) The degree to which the foreign language was used in the classroom 

by the teacher. 

(2) The degree to which the student was required to use the FL. 

' 

(3) The degree to which a language laboratory was used, and if so, the extent 

to which it was an Integral part of the course . 

(4) The extent to which the teacher was judged to have a native or near-native 

accent in the foreign language. 

Each of these variables was averaged over groups of courses — high school courses, and 
college beginning and intermediate language courses. Some results obtained for these 
variables are shown in Table 7.7. The first part of this table concerns groups of stu- 
dents who started. FL study in high school, and therefore it presents data concerning in- 
structional variables applicable at that stage. If we look at means, we notice the 
following about high school foreign language instruction as reported by the students: 

1. Teaching was mostly In English. 

2. In the main, the student's classroom language was English. 

3. Generally, there was very little use of language laboratories. 

4. The average teacher's ability to pronounce the FL correctly was about half- 

way between extremes of Incompetence and the competence of a native spealcer. 
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Table 7.7 

Data for Certain Instructional Variables 
an/^ Correlations with MLA Skills Tests 



A. Group Starting In High School Correlations with MLA Skills Test 

T Moon S .D. T. S R W 



Variable Lang. N 

Teacher's use of FL, h.s. F 733 


Mean 

1.74 


S.D. 

0.60 


L 

.09* 


S 

.06 


R 

.02 


W 

.06 


f 


l"Mostly English used ) 


G 131 


1.61 


0.64 


.20* 


.19* 


.20* 


.18* 


Coding : S 


3"Mostly FL used j 


S 519 


1.73 


0.65 


.12** 


.13** 


.13** 


.14** 


Student use of FL, h.s. 


F 733 


1.29 


0.40 


.09* 


.07* 


.04 


.06 


Coding: | 


laMostly English used ) 


G 131 


1.20 


0.35 


.24** 


.21** 


.22* 


.17 


2*Mostly FL used ' 


S 519 


1.26 


0.38 


.08 


.09* 


.08 


.14** 


language lab, importance. 


F 733 


1.22 


0.44 


.06 


.02 


.02 


.03 




h.s. courses 
















Coding: 


f l«No language lab. 1 


G 131 


1.22 


0.47 


.08 


.00 


.15 


.09 


. >Lang.Lab. important- and 
Integral part ^ 


* S 519 


1.22 


0.49 


.05 


.02 


.08 


.06 


Teacher's pronunciation, 
h.s. courses 


F 733 


1.95 


0.70 


.04 


.08* 


.00 


.02 




' l»Deflv-ite non-native "j 








.16 


.13 


.18* 


.11 


Coding: 


accent 7 


G 131 


2.22 


0.74 


. 3"Hatlve accent j 


S 519 


2.11 


0.68 


.04 


.05 


.02 


.00 




Group starting In .^ollege 








Correlations with MLA Skills Test 




Variable 


Lang. H 


Mean 


S.D. 


L 


S 


R 


W 


Teacher's use of FL, coll. 


F 167 


2.20 


0.50 


.27** 


.27** 


.17* 


.18* 




lang. courser 














.22** 


Coding : 


(■ 1-MoStly English used 1 


G 151 


2.08 


0.58 


.23** 


.14 


.14 


( 3"Mostly FL used j 


R 58 


2.17 


0.53 


.07 


-.01 


.05 


.08 






S 154 


2.24 


0.53 


.39** 


.32** 


.35** 


.36** 


Student use of FL. college 








.21** 


.19* 


.13 


.15 




lang. courses 


F 167 


1.54 


0.31 


Coding : 


( l«Mostly English used ) 


G 151 


1.47 


0.35 


.30 


.20 


.20 


.28 


( 2"Mostly FL used ' 


R 58 


1.61 


0.31 


.29* 


.20 


.29* 


.25 






S 154 


1.53 


0.32 


.32** 


.29** 


.26** 


.28** 




Language lab. Importance, 
coll. lang. courses 


F 167 


1.88 


0.54 


-.03 


-.03 


-.03 


-.13 


Coding ; 


( l«Ho lang. lab. ? 


G 151 


1.81 


0.57 


-.05 


-.15 


-.08 


-.14 


( 3-Lang. Lab. Important ) 


R 58 


1„94 


0.56 


-.02 


-.02 


-.03 


.05 






S 154 


1.72 


0.58 


-.11 


-.05 


**•03 


-.14 




Teacher's pronunciation^ 
coll. lang. courses 


F 167 


2.56 


0.47 


.17 


.14 


.13 


.15 




/ l"Def . non-native accent 


7 G 151 


2.65 


0.42 


.18* 


.07 


.12 


.17* 


Coding 


( 2*^Natlve accent 


Jr 58 


2.70 


0.41 


-.07 


-.07 


-.14 


-.08 






S 154 


2.56 


0.42 


.10 


.12 


.12 


.13 
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On the other hand» thei? was considerable variation in students' experiences. Some stud- 
ents had apparently had the benefit of teaching that employed much active foreign lang- 
uage use by both teache and student. Although it was a little too much to hope that such 
variations would be associated with performance on the MLA Skills tests taken at least 4 
years after the high schiol experience* Table 7.7 actually shows a number of significant 
correlations* and these are more or less consistent across languages. The teacher's use 
of the FL in the classro:>m and the student's use of it are about equally correlated with 
MLA skills test scores. The excellence of the teacher's pronunciation does not seem to 
make much difference* and variations in emphasis on language laboratory use do not corre- 
late at all with performance on the MLA tests. (The latter finding may laflect the fact 
that good teaching may le done either with or without a language laboratory.) 

The second part of the table concerns groups that started study of the FL in college* 
and the instructional variables apply to college courses only. The mean scores for the 
variables indicate that instructional procedures in college differ markedly from the high 
school* at least according to these students' reports. Much more use of the foreign 
language is made both bv' the teacher and the student* and teachers' pronunciations are 
more likely to be of a i, atlve or near-native character. There is considerably more 
emphasis on the use of language laboratories* but it should be noted that wheths^.r a stud- 
ent has had a language laboratory experience has little or nothing to do with his event- 
ual attainment of skill: indeed* the trend is toward a negative (but never significant) 

correlation. But teacher and student use of the FL in the classroom is frequently sig- 
nificantly associated with tested skill. 

It should be borne in mind that MLA skill scores are associated with many variables 
besides instructional ones* as we have seen and shall see; the fact that statistically 
significant correlations were obtained at all for these instructional variables does bear 
testimony* however* for the proposition that quality of instruction has at least some- 
thing to do with student attainment. When quality of instruction* as measured by the 
instructional variables mmtloned here* is combined appropriately with information on 
student aptitude* amount of exposure to the language* and other variables* it is possible 
to predict attainment more accurately than otherwise* as postulated by the writer in a 
previous publication (Carroll* 1963a). 

4. Time spent abroad 

On page 4 of the Questionnaire for Foreign Language Majors (Appendix B)* students 
were asked to indicate what kind of experience they may have had in travelling or study- 
ing in the country where their major language is spoken natively. On the basis of the 
responses* students were classified into the following groups: 

1. Students who had had a school year abroad ("a regular school year in a country 
where your major language is spoken natively and took a series of courses in your 
major language during that time") 

2. Students who had had a summer of study abroad (but not a year abroad) 

3. Students who had had neither a school year nor a summer of study abroad but who 
had travelled as a tourist in the major -language country and/or had had a program 
of self-study there 

4. Students who had never been abroad to the major-language country. 

Table 7.8 shows the numbers of such students* by language and by sex* al /ng with mean MLA 
skills test scores for each sub-classification. 

Substantial numbers of students had been abroad to a country where their major lang- 
uage was spoken: 51.5% of the French majors* 60.1% of the German majors* 22.6% of the 

Russian majors* and 55.3% of the Spanish majors. There was no significant difference be- 
tween men and women in this respect for French and German; however* significantly more 
men (36.7%) than women (14.8%) had been abroad in the Russian group* while significantly 



o 




05; §not computed because of small N 
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more women (56.9Z) than men (50.5%) had been abroad in the Spanish major group. 

In MLA skllle test scores • there were highly significant differences In every lang- 
uage group among groups with different amounts of experience abroad, favoring those who 
had been abroad, and particularly those who had been abroad for a year of study. Persons 
who had had only a summer abroad, or a tour, had mean test scores Intermediate between 
those who had had a year abroad and those who had never been abroad. Since It did not 
seem ,,;o make much difference In these intermediate groups whether the experience abroad 
was a summer or a tour, these groups were combined for subsequent analyses, and the Time 
Abroad variable was re-coded so as to constitute three categories: 

3: A year abrroad 

2: Summer study or a tour abroad 

1: Never abroad 

As stated in the opening paragraphs of this chapter, it Is temp ing to conclude that 
golnF, abroad Is an Important causative variable influencing language competence, and such 
a conclusion accords with common experience. Indeed, the highly significant effects re- 
ported above are not likely to have occurred simply as a result of selective factors. 
Nevertheless, until more information is available from studies based cn pre-test and post- 
test comparisons of groups who are and who are not sent abroad. It Is Impossible to state 
with certainty— at least on the basis of the present data— just what the effect of travel 
and study abroad may be. Carton and Carroll (1960) gave pretests and posttests to groups 
if Russian students making a six-week tour of the Soviet Union and concluded that the 
tour yielded distinctly Increased competence, especially in speaking skills, and especi- 
ally for student*! with at least two years' prior study of Russian, but they were not able 
to compare the effects of the tour with other kinds of language training experiences. 

The effect of Time Abroad is Independent of Time of Beginning FL Study, as shown by 
Table 7.9 for the Listening test scores for French, German, and Spanish groups. The F- 
ratios for the Interaction of the two variables are, respectively, .2C, .10, and .26- 
all Insignificant. Similar results, not shown here, were obtained for the Speaking, Read- 
ing, and Writing tests. Incidentally, there are significant tendencies shown In Table 
7.9 for persons who started their language study early to have had more experience abroad 
than those who started their language study later. In French, the percentages of students 
who have had at least some experience abroad are as follows: grade school starters, 

59.7%; high school starters, 50.3%; college starters, 50.3%. In Spanish, the percentages 
are, respectively, 69.2%, 52.9%, and 50.0%. However, the trend Is reversed In German: 
only 49.6% of the high school starters had been abroad, while 60.9% of the college start- 
ers had. The findings in French and Spanish possibly reflect economic factors whereby 
those having FLES are more likely to have come from families who could afford to send 
them abroad. The lack of significant interactions in the effects of the two variables on 
MLA skills test scores suggests, however, that the students who traveled abroad after 
FLES experience did not gain any special advantage thereby. (That Is, the fact that they 
had had FLES experience did not make them more susceptible to Improvement by foreign 
study than students without such FLES experience.) 

Time Abroad appears to be a more influential variable than Time of Beginning FL 
Study, to judge from the fact that the former shows wider separation of means by sub- 
classifications . 

In none of the languages were the groups classified by amount of time abroad sig- 
nificantly different on any of the MLAT (language aptitude) scores. Going abroad thus 
does not appear to depend on language aptitude as a self-selection factor, and it does 
not affect MLAT scores, according to these results. 
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Table 7.9 



Mean Listening Test Scores (and N^s) 
by Time of Beginning FL Study a*id by Time Spent Abroad 









Time Began 










Grade School 


High School 


College 


Total 








FRENCH 








1 


Year Abroad 


51.74 (38) 


50.22 


(160) 


47.52 


(31) 


50.11 


(229) 


Tour or Summer Abroad 


48.16 (45) 


45.55 


(209) 


42.15 


(53) 


45.34 


(307) 


Never Abroad 


43.70 (56) 


40.78 (364) 


38.04 


(83) 


40.65 


(503) 


Total 


47.34 (139) 


44.20 


(733) 


41.10 


(167) 


44.12 


(1039) 








GERMAN 










Year Abroad 


(Not used 


50.03 


(30) 


50.65 


(37) 


50.37 


(125) 


Tour or Summer Abroad 


In compu- 


44.71 


(35) 


45.89 


(55) 


45,42 


(90) 


Never Abroad 


tations) 


36.95 


(66) 


37.27 


(59) 


37.10 (125) 


Total 


- 


42.02 


(131) 


43.69 


(151) 


42.91 


(282) . 








SPANISH 








Year Abroad 


52.55 (29) 


50.08 


(73) 


47.92 


(13) 


50.46 


(115) 


Tour or Summer Abroad 


48.53 (34) 


45.10 


(201) 


44.75 


(64) 


45.42 


(299) 


Never Abroad 


45.21 (28) 


41.92 


(244) 


39.96 


(77) 


41.75 


(349) 


Total 


48.79 (91) 


44.30 


(518) 


42.62 


(154) 


44.50 


(763) 
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5. Use of the Major FL at Home 

Students were asked, "Do you and/or your parents speak your major language at home?" 
and were allowed to check either "No," "Yes, occasionally," or "Yes, frequently. The 
ambiguity of the question is perhaps unfortunate, because we cannot tell whether the 
student himself used the foreign language with his parents. The question was intended 
merely to reveal whether the student was exposed to use of the foreign language n . s 
home. The means of MLA skills test scores for students answering each alternative are 
shown, for each language (except Italian), in Table 7.10. Highly significant 
differences are nearly everywhere apparent, favoring those with frequent use of the FL at 
home. The groups used in the analysis were "regular" cases only; students identified as 
native speakers of the language in question had been excluded. Those who cam requen 
use of the FL at home either by themselves or by their parents do almost as well as stud- 
ents classified as native speakers (see Chapter IV). Obviously home is an excellent 
place to learn a foreign language if it is regularly spoken there. There are small num- 
bers of students in our sample who had that opportunity, and substantial nunbers whose 
homes were ones where the FL was used "occasionally." This fact must be held in mind in 
interpreting norms presented in Chapters IV ^d V. 



6. Other Experiences in the Foreign Language, 



Students were asked, "Considering only your major language, have you taken infora- 
aiiy (not for credit) any course, or audited any course, in that language (whether 
summer, night, special, or other)?" Of 1114 French "regular" cases, 29. IZ indicated 
"yes" to this question. The figures for the other languages are: of 324 German stud- 

ents, 29.6% "yeses"; of 88 Russian students, 19. 3Z; of 814 Spanish students, 24. 9Z. For 
French, German, and Spanish students, highly significant differences were found on each 
of the MLA skills tests, favoring those who reported having taken Informal courses. We 
can only guess at the nature of these courses. The fact of having taken an informal 
course may be an indication of special motivation on the part of the student; it uoes 
not necessarily represent an Influence on the student. 



Another question asked was, "Have you had occasion to use your major language in a 
job situation, with foreign friends, or in some other extracurricular situation? Such 
opportunity was probably not independent of other experiences in the backgrounds of the 
students, and it is not clear whether this variable should be regarded as a predictor 
variable or a criterion variable, because this "extracurricular" use of a foreign lang- 
uage might signify either a cause or an effect. At any rate, large numbers of students 
answered either "moderately" or "extensively" to this question, as may be seen 
7.11, and there were in nearly every case highly significant differences in mean 
skills test scores among the different response groups, always favoring those with 
"extensive" outside use of the language, and to a lesser degree, those with ’moderate 
outside use, over those reporting no outside use. 



The amount of independent reading that the student had done was assessed by the 
question, "Independent of and in addition to reading required in courses, have you had 
any other reading experiences in your major language?" The responses to this question 
constitute another variable which cannot clearly be classified as either an antecedent or 
a consequent of fortign language attainment, but it was of interest to learn how much 
Independent reading the students had done and to see whether it was associated with test- 
ed achievement. Table 7.12 reports the numbers of students in each languap who indl- 
cated each of the possible answers to the question given on the questionnaire: 'have not 

done any reading except for material in course work"; "read a few pages of materfal ; 
"read one or two books (or the equivalent)"; "and "read three or more books. The dis- 
tributions of responses vary according to the sex of the respondent and also from ang- 
uage to language. There is a tendency for the men to have read more widely than the 
women. Students of French tend to do much more independent reading than students of 
other languages; 55.3% of them report having read three or more books. 47. 8Z of the 
German students had read three or more books, 42.1% of the Spanish students, and only 
22.7% of the Russian students, who, it will be recalled, tend to do relatively worse on 
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Table 7.10 

Mean MLA Skills Test Scores by Use of the FL at Home 

(''Regular" Cases Only) 




Use of FL at home 


N 


% 


Mean 

Listening 


MLA Skills 
Speaking 


Test Scores 
Reading Writing 








FRENCH 






Frequently 


15 


1.3 


49.40 


83.67 


53.20 


53.53 


Occasionally 


101 


9.1 


46.53 


77.18 


50.57 


49.25 


Never 


998 


89.6 


43.78 


73.03 


48.56 


46.84 


Total 


1114 


100.0 










F-ratio 






9.8** 


18.9** 


4.5* 


6.4* 








GERMAN 






Frequently 


14 


4.3 


51.71 


101.64 


59.71 


63.07 


Occasionally 


49 


15.1 


46.14 


92.29 


52.18 


54.57 


Never 


261 


80.6 


42.49 


84.91 


50.34 


51.03 


Total 


324 


100.0 










F-ratio 






11.1** 


18.5** 


6.7** 


6.6** 








RUSSIAN 






Frequently 


1 


1.1 


52,00 


104.00 


50.00 


71.00 


Occasionally 


7 


8.0 


44.43 


83.71 


45.00 


65.14 


Never 


80 


90.9 


42.28 


77.55 


40.85 


61.72 


Total 


88 


100.0 











No test of significance was run for Russian 



SPANISH 



Frequently 


49 


6.0 


50.14 


97.59 


55.45 


61.22 


Occasionally 


75 


9.2 


47.84 


89.71 


50.83 


57,59 


Never 


690 


84.8 


43.70 


82.43 


46.95 


53.81 


Total 


814 


100.0 










F-ratio 






33** 


51** 


31** 


16** 
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Table 7.11 



Amount of Use of the FL in Jobs, with Friends, Etc., 

by Language and by Sex 



Amoimt of FL use outside 


N 


Men 

% 


N 


Women 

% 


N 


Total 

% 






FRENCH 










Extensive 


31 


17.5 


133 


14.2 


164 


14.7 


Moderate 


98 


55.4 


491 


52.4 


589 


52.9 


None 


48 


27.1 


313 


33.4 


361 


32.4 


Total 


177 


100.0 


937 


100.0 


1114 


100.0 






GERMAN 










Extensive 


28 


21.1 


37 


19.4 


65 


20.1 


Moderate 


65 


48,9 


101 


52.9 


166 


51.2 


None 




30.1 


53 


27.8 


93 


28.7 


Total 


133 


100.1 


191 


100.1 


324 


100.0 






RUSSIAN 










Extensive 


4 


12.1 


3 


5.4 


7 


7.9 


Moderate 


11 


33.3 


30 


54.5 


41 


46.6 


None 


M 


54.5 


22 


40.0 




45.4 


Total 


33 


99.9 


55 


99.9 


88 


99.9 






SPANISH 










Extensive 


40 


19.8 


124 


20.3 


164 


20.1 


Moderate 


120 


59.4 


345 


56.4 


465 


57.1 


None 


42 


20.8 


Ml 


23.4 


185 


22.7 


Total 


202 


100.0 


612 


100.1 


814 


99.9 
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Table 7.12 

Amount of Independent Reading, by Language and by Sex, with 
Mean Reading Test Scores for Language Groups 



Amount of 

Independent reading 




Men 


Women 


N 


Total 


Mean 

Reading 

Score 


N 


% 


N 


% 








FRENCH 










None 


7 


4.0 


38 


4.0 


45 


4.0 


44.64 


Little 


17 


9.6 


147 


15.7 


164 


14.7 


43.42 


One or two books 


45 


25.4 


244 


26.0 


289 


25.9 


46.29 


Three or more books 


108 


61.0 


508 


54.2 


616 


55.3 


49.66 


Total 


177 


100.0 


937 


99.9 


1114 


99.9 


F^31** 








GERMAN 










None 


9 


6.8 


7 


3.7 


16 


4.9 


43.75 


Little 


25 


18.8 


27 


14.1 


52 


16.0 


43.48 


One or two books 


29 


21.8 


72 


37.7 


101 


31.2 


47.56 


Three or more books 


70 


52.6 


85 


44.5 


155 


47.8 


56.57 


Total 


133 


100.0 


191 


100.0 


324 


99.9 


F^50** 








RUSSIAN 










None 


13 


39.4 


21 


38.2 


34 


38.6 


36.6 


Little 
















One or two books 


13 


39.4 


21 


38.2 


34 


38.6 


41.3 


Three or more books 


_7 


21.2 


11 


23.6 




22.7 


49.2 


Total 


33 


100.0 


55 


100.0 


88 


99.9 


F*13.6** 








SPANISH 










None 


14 


6.9 


33 


5.4 


47 


5.8 


43.4 


Little 


40 


19.8 


149 


24.3 


189 


23.2 


43.6 


One or two books 


52 


25.7 


183 


29,9 


235 


28.9 


45.9 


Three or more books 


96 


47.5 


247 


40.4 


343 


42.1 


52.0 


Total 


202 


99.9 


612 


100.0 


814 


100.0 


F=66** 



**p < .01 
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the tests than students of other languages, and have had relatively less exposure to 
formal Instruction. Table 7,12 also shows the mean Reading test scores for the groups of 
respondents according to the amount of Independent reading they have done. Students who 
report having done no Independent reading are at about the same level of achievement as 
those who say they have read "only a few pages" of material In the foreign language. As 
the amount of Independent reading Increased beyond "a few pages," the mean Reading test 
scores Increased monotonlcally . Students who have read three or more books attain dis- 
tinctly superior Reading test scores. (Scores on the other MLA skills tests show similar 
trends but are not reported here.) 

7. Other Foreign Languages Taken Besides Major Language 

From data supplied by the students. Table 7.13 has been constructed to show the fre- 
quency with which majors in a given language had studied other modern foreign languages; 
the cases were classified according to the time they began study of their major language. 
Information was not obtained as to the tlme(s) the students studied the other languages. 
Common observation would suggest that those who started their major language In grade 
school would most likely have studied other languages at a later time, while those who 
started their major language in college are most likely to have started another language 
at an earlier time. There Is nothing In the table to suggest that those who started 
their major language In grade school were any more likely than those who started In high 
school or college to have had study of other languages. 

As one might expect, students majoring In a given Romance language are fairly likely 
to have studied other Romance languages. Of the French majors, 45 « 9% have studied Span- 
ish, and 13.3% have studied Italian. 63.2% of the Italian majors hav^ studied French and 
17.4% Spanish. 56.9% of the Spanish majors have studied some French, and 10.8% of them 
have studied Italian. It may be noted that although Italian Is not very popular as a 
major. It Is nevertheless studied by substantial numbers of students, mostly majors In 
other Romance languages. 

French Is a very popular second language even for majors In German and Russian. 

German Is particularly popular with majors In Russian, and to some extent also with majois 
In French. Russian is a second foreign language studied by small numbers of majors In 
other languages; It Is most popular among students of German. Spanish Is studied by ap- 
preciable numbers of German and Russian mfxjors, along with the students of Romance 
languages . 

Considerable numbers of all the groups Indicated that they had studied other modem 
foreign languages besides the 5 test languages. Tliere was a great variety of such lang- 
uages and no tabulations are presented here. 

There has been perennial interest in the question of whether the study of one for- 
eign language helps in the study of another language. The present Investigation was not 
designed to study this question with any rigor, although with careful statistical analysis 
the data might permit certain tentative conclusions. Table 7,14 represents a first step 
towards such an analysis; it shows the mean MLA skills test scores of those who have had 
study of other languages in comparison to the means for the complete language major 
groups. The comparisons were tested for statistical significance by the ^-test of the 
difference in the means of those who had had the other language and of those who had not, 
A number of comparisons were highly significant, usually favoring those who had had study 
of another language besides their major. For example, French majors who had also had 
study of German, Italian, or Russian were without exception significantly superior on 
tests to those who had not studied the given other language, German majors who had had 
French were superior to those who had not, but German majors with Spanish study were in- 
ferior on the Listening and Reading tests to those without Spanish study. A few compari- 
sons In the Spanish group were highly significant. 

It is very difficult to Interpret these results at face value. There does not seem 
to be any consistent pattern whereby study of related languages (in the Romance group) 
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Table 7.13 

Other Languages Taken, By Time of Beginning Major Language 
(Excluding Latin and Ancient Greek) 



Time of ag inning 
Study of Major 
Language 



Language other than the Major Language 



French 
N % 



German 
N % 



Italian 
N % 



Russian 
N % 



Spanish 
N % 



Other 
N % 



FRENCH MAJORS 



Grade school 


(148) 




High school 


(784) 




College 


(1821 




Total 


(1114) 




Grade school 


6 


85.7 


High school 


66 


44.9 


College 

Total 


96 


56.5 


168 


51.8 


Grade school 


0 


— 


High school 


4 


50.0 


College 


11 


84.6 


Total 


15 


65.2 


Grade school 


1 


50.0 


High school 


14 


63.6 


College 


41 


64.1 


Total 


56 


63.6 


Grade school 


70 


72.9 


High school 


303 


54.6 


College 

Total 


90 


55.2 


463 


56.9 


Grade school 
(N = 255) 


77 


30.2 


High school 
(N = 1516) 


387 


25.5 


College 
(N = 592) 


238 


40.2 


To^al 

(N » 2363) 


702 


29.7 



31 

188 

58 


20.9 
24.0 

31.9 


29 19.6 

102 13.0 

17 9.3 


17 

81 

10 


11.5 

10.3 

5.5 


70 

348 

93 


47.3 

44.4 
51.1 


277 


24.9 


148 13.3 


108 


9.7 


511 


45.9 




GERMAN MAJORS 










( 7 ) 




0 — 


0 


0.0 


2 


28. e 


(147) 




11 7.5 


20 


13.6 


32 


21. ( 


(170) 




7 4.1 


28 


16.5 


55 


32.^ 


(324) 




18 5,5 


48 


14.8 


89 


27.1 




ITALIAN MAJORS 










0 


— 


( 2) 


0 


— 


1 


50. ( 


0 


— 


( 8) 


0 


— 


1 


12.: 


0 


— 


im. 


0 


— 


2 


15.' 


0 


— 


(23) 


0 


— 


4 


17. 




RUSSIAN MAJORS 










0 


— 


0 


( 2) 


0 


— 


10 


45.4 


0 — 


(22) 


3 


13. 


26 


40.6 


5 7.8 




17 


26. 


36 


40.9 


5 5.7 


(88) 


20 


22. 




SPANISH MAJORS 










13 


13.5 


11 11.5 


4 


4.2 


( 


96) 


77 


13.9 


64 11.5 


38 


6.8 


(555) 


28 


17.2 


13 8.0 


10 


6.1 


(163) 



52 6.4 



(814) 



118 14.5 88 10.8 

ALL MAJORS 

77 30.2 44 17.2 40 15.7 21 8.2 73 28.6 88 34.5 

25.5 275 18.1 177 11.7 139 9.2 384 25.3 523 34.5 

23 8 40.2 112 18 . 9 42 7.1 48 8.1 167 28 ._2 261 — 44 .1 , 

29.7 431 18.2 259 11.0 208 8.8 624 26.4 872 36.9 



53 


35.8 


294 


37.5 


72 


39.6 


419 


37.6 


4 


57.1 


51 


34.7 


82 


48.2 


137 


42.3 


0 




1 


12.5 


5 


38.5 


6 


26.1 


1 


50.0 


12 


54.5 


32 


50.0 


45 


51.1 


30 


31.2 


165 


29.7 


70 


42.9 


265 


32.6 


88 


34.5 


523 


34.5 


261 


44.1 


872 


36.9 



0 






“147- 

Table 7.14 

Mean MLA Skills Test Scores for Groups with Study of Other Languages 

Besides their Major Languages 





N 


Mean MLA Skills 
Listening Speaking 


Test Scores 
Reading 


Writing 








FRENCH 






Had German 


277 


46.03** 


74.84** 


50.89** 


49.18** 


Had Italian 


148 


46.52** 


75.47** 


51.94** 


50.93** 


Had Russian 


108 


47.21** 


76.22** 


53.48** 


52.30** 


Had Spanish 


511 


44.28 


74.12 


48.98 


47.54 


Had other langs. 


419 


44.14 


72.99 


49.28 


47.45 


All French 


1114 


44.11 


73. 5 j 


48.81 


47.15 








GERMAN 






Had French 


168^ 


44.62* 


88.82** 


52.18* 


53.79* 


Had Italian 


18 


44.17 


92.11 


52.61 


57.22 


Had Russian 


48 


45.42 


88.67 


53.71* 


56.29* 


Had Spanish 


89 


41.39**(- 


) 84.57 


48.78*(-) 


50.30 


Had other langs. 


137 


43.67 


87.28 


52.03 


53.73 


All German 


324 


43.44 


86.75 


51.03 


52.09 








RUSSIAN 






Had French 


56 


42.71 


78.12 


41.12 


62.41 


Had German 


37 


42.62 


80.35 


41.97 


64.22 


Had Italian 


5 




(Not computed) 




Hac| Spanish 


20 


41.45 


76.90 


39.00 


61.90 


Had other langs. 




42.40 


78.42 


40.87 


62.53 


All Russian 


88 


42.56 


78.34 


41.28 


62.10 








SPANISH 






Had French 


463^ 


45.67** 


85.58 


48.99 


56.04** 


Had German 


118 


44.68 


83.35 


49.08 


55.39 


Had Italian 


88 


45.54 


85.26 


50.33** 


57.04* 


Had Russian 


52 


46.04 


88.27* 


50.52* 


59.00** 


Had other langs. 


265 


44.54 


83.17 


47.80 


55.13 


All Spanish 


814 


44.47 


84.01 


47.82 


54.60 



^These comparisons were also analyzed with covariance adjustment for language aptitude 
(MLAT total); the differences remained significant. 

**p < .01, *p < .05 for comparison with means for those not having had study of the 
language in question. A minus (-) attached calls attention to the fact that the mean is 
lower than that of the comparison group. 
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is more associated with higher test scores than study of languages that are less related 
(Romance vs. Germanic vs. Slavic). It is possible that the comparisons reflect relative 
amounts of time spent on the respective languages, differences in motivation for language 
study, etc. For example, the lower scores of German majors who have had Spanish might re- 
flect a lesser amount of study of German due to increased time devoted to Spanish study. 
Although (as mentioned above) the data of the present investigation could afford further 
analyses, it has not been considered worthwhile to pursue these analyses for the present 
report partly because it was felt that even these further analyses would lead to more 
questions than they could answer, in view of the lack of data on when and how long the 
other languages had been studied. 



Somewhat more effort, however, has been expended on trying to analyze the effect of 
prior study of Latin and Greek on performance on the MLA tests, since the questionnaire 
provided specific data on when these langusges had been studied (high school or college) . 
It was realized that any analysis should take account of not only when the study of Latin 
(or ancient Greek) was taken up but also the time of beginning the major language. Fur- 
ther, language aptitude would have to be controlled, it least statistically. Therefore, 
in each of the three languages in which there were sufficient numbers of cases (French, 
German, and Spanish) , a two-way analysis of variance was performed with Time of Beginning 
the major language and Time of Study of Latin (or Greek) as the main effects and with co- 
variance adjustment for the total score (for parts 3-5) of the MLAT. The principal re- 
sults are shown in Table 7.15. Data are given only for the Latin-No Latin main effect; 
although the Time of Beginning the major language produced highly significant effects, 
these are similar to those discussed earlier in this chapter and need not further concern 
us. The interaction effects between the two main effect variables were all insignificant. 



The relevance of the covariance adjustment for language aptitude becomes apparent 
when the mean MLAT scores are examined for the groups with different experiences in Latin. 
It is evident that MLAT is correlated to a considerable extent with Latin study, probably 
significantly so. (This particular finding was not separately tested because it is not of 
direct concern to this investigation.) There is no sure explanation for this; the most 
probable is that students tend to self-select themselves for Latin study on the basis of 
their experiences with foreign languages, which are to some extent related to language 
aptitude. It is also possible that the study of Latin or ancient Greek may enhance some 
of the abilities intrinsic to language aptitude as measured by the MLAT. 

In the table, the mean MLA skills test scores for the various groups are given with- 
out adjustment for language aptitude. At the right of the table, however, are given the 
"effects” of the Latin-No Latin variable after covariance adjustment for MLAT. These ef- 
fects are the deviations of the adjusted mean scores, measured from the grand means of 
the relevant variable, associated with each sub— classification of the Latin- No Latin vari- 
able. The effects for any given sample and variable sum to zero. For example, for the 
French group, under the Listening column, the effects are rO.35, 0.91, 0.47, and —1.03. 
This means that the No Latin group’s adjusted mean Listening score is .35 points under the 
adjusted grand mean, the High School Only group’s adjusted mean score is .91 points above 
the adjusted grand mean, etc. That is, for this variable, having had no Latin depresses 
the mean score while having had Latin in high school raises it. 



In the whole table, there is only one series of effects that are statistically sig- 
nificant (i.e. beyond chance fluctuations) at the 1% level, namely the effects for the 
Spanish Speaking score. Here, having had no Latin enhances the Speaking score, while 
having had it in high school or college depresses it. This finding would tend to support 
the hypothesis held by some foreign language teaching experts that exposure to the grammar 
-translation practices of most Latin teaching tends to transfer negatively to the audio- 
lingual language learning desired in modern language teaching. The hypothesis is based on 
the notion that granauar-translation methods incline the student to make too much refer(?,nce 
to English (that is, to the native language) and thus inhibit the development of success- 
ful foreign-language study habits in contexts stressing the making of immediate foreign- 
language responses without reference to the native language. It is interesting that this 
significant effect occurs only in the Speaking test, which presumably demands fluent for- 
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Associated with the Study of Latin and/or Ancient Greek 
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eign-language production; in the same group, the effect is reversed for the other tests 
(Listening, Reading, and Writing). The effect for the Writing test is nearly significant 

at the 5% level. 

On the other hand, no such effect for the Speaking test occurs for the other lang- 
uages. In French, which is cognate with Spanish, the Speaking test results show greatest 
positive effect of Latin study for those who studied it in college. The only agreement 
between the Speaking test results in French and in Spanish lies in the fact that in both 
cases there is a negative effect for the group that studied Latin in high school—again 
supporting the thesis mentioned above that early exposure to Latin may inhibit good mod- 
em foreign-language learning. 

In all three languages, positive effects of Latin study are to be noted in the Writ- 
ing test scores for those who studied Latin in college; the effect is statistically sig- 
nificart in French and nearly so in German and Spanish. Since the Writing test stresses 
gram^’ar, this finding would support the idea that Latin study has at least some beneficial 
effeC'^ on those aspects of modern language study that, are associated with grammatical 
analysis, thus reinforcing the arguments that teachers of classical languages have been 
advancing for many years. 

On the whole, however, the effects of Latin study revealed in this analysis are 
rather slight — seldom approaching statistical significance. In contrast to variables such 
as Time of Beginning FL Study and Amount of Time Abroad, it is a very weak and unreliable 
influence. There is little point in trying to interpret MLA skills test scores with re- 
ference to the amount of Latin study the individual has had. 
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Chapter viii 



REGRESSION ANALYSES FOR PREDICTING MLA SKILLS TEST SCORES 
FROM COMPOSITES OF VARIABLES 



1. Introduction 

In Chapters VI and VII we have explored relationships between MLA skills test scores 
and a number of student and instructional variables that were thought to be possibly as- 
sociated with them. Each such variable was taken up in turn. A drawback of that proced- 
ure was that the possible interrelationships among the associated background variables 
were generally not taken into account. If two such variables are strongly correlated, 
redundancy exists and one variable should not be considered without the other. The pres- 
ent chapter attempts to remedy this drawback by utilizing statistical techniques whereby 
oW the interrelationships of relevant background variables are examined* The goal is to 
determine how well the MLA test scores can be "predicted” or "explained" by the background 
variables. To the extent that background variables are amenable to manipulation— e.g. by 
sending more students abroad, or by improving certain aspects of instruction, the results 
of these analyses may suggest ways of turning out better language students. 

The techniques of analysis come under the heading of what is called linear multivari- 
ate analysis. Multiple regression and canonical correlation are the two techniques that 
are employed extensively in this chapter. We shall attempt to explain these techniques 
for the layman by carefully describing their application to some relatively simple tabu- 
lations developed in the present study. First, however, several statistical terms will 
be Introduced and defined: 

Predictor: a variable that is used as a basis for prediction* For example, we can 

use scores on the MLAT as a possible predictor of foreign language achievement as measured 
by one of the MLA skills tests. Use of the term "predictor" does not necessarily imply 
that the variable is actually efficient as a predictor; it merely implies that the vari- 
able is entered into a regression system (defined below) as a possible predictor. 

Criterion (plural: criteria) : a variable that one hopes to be able to predict* In 

the present study, the MLA skills test scores serve as the criteria, since we hope to 
"predict" their values for individual cases, and thus to "explain" their variation. 

Correlation : a statistical measure of the association between tvi^o variables, e.g* 

between a predictor and a criterion. When the correlation coefficient is zero, it indi- 
cates no association between the variables in question. When it is unity (+1*00), it 
indicates that the two variables are perfectly correlated, that is, that every individual 
value of one of the variables is relatively just as high or low as the corresponding value 
on the other variable. When it is negative unity (-1.00), it indicates that the values 
are inversely correlated, that is, that each individual value of one of the variables is 
just as relatively high as the corresponding individual value of the other variable is 
relatively low * and vice versa . Tlie simple correlation coefficient may take values any- 
where between +1 and -1. Correlations are computed on the basis of a series of paired 
values. When a correlation is computed between a predictor and a criterion, it is some- 
times called a validity coefficient. Often, one is interested in determining the statis- 
tical significance of a correlation, that is, in whether the actual correlation departs 
significantly from the range of values that one might obtain by chance, because any series 
of values paired at random will show some tendency to be correlated, positively or nega- 
tively. Generally one tests the significance of a correlation coefficient as departing 
from the value of .00 that would be postulated to occur, on the average, if there is not 
truly an association between the variables. 

Regression : When two variables, X and Y, are imperfectly correlated, that is, with 

the absolute (non-signed) coefficient less than 1.00, it will be found that the values of 



mmm 



-152- 



Y associated with a given value of X tend not to be as far, relatively, from the mean of 

Y as the value of X is from the mean of X for all cases. Thus, values of Y are said 

to regress toward the mean. The average degree to which this regression occurs can be de- 
nlctad bv“ a regression line joining the average or expected values of Y for given values 
of X. The slope of the regression line is a function of the degree of correlation. For 
the procedure to be used here, it is implied that the regression line is the straight line 

of best fit to the data. 



Multiple regression: a generalization of simple regression to the case where Y is 

predicted from ^ or more variables, X^, X^, etc. The regression line becomes, geomet- 
rically, a plane (in the case of two predictor variables) or a hyperplane (for three or 
more predictor variables), and the slopes of the plane or hyperplane can be measured with 
respect to each one of the predictor variables. 



Regression analysis : the process of computing the best possible prediction line, 

plane, or hyperplane, and its slopes with respect to the predictor variables. 

Regression system : the results of regression analysis for a given set of data, in 

eluding the correlations, regression coefficients, etc. 



Multiple correlation: a coefficient, analogous to the simple correlation coeffici- 

ent, that expresses th^e degree to which given data are approximated by the best regression 
line (plane, hyperplane) computed for the data. It measures, therefore, the extent to 
which a given criterion variable can be predicted from a series of predictor variables. 

The multiple correlation coefficient may range between 0 and 1. 



Beta coefficient or Beta-weight : technically, a measure of the slope of the regress- 

ion plane with respect to a given predictor variable (all variables being expressed in 
standardized form) . It indicates the degree to which the given predictor variable unique- 
ly contributes to the prediction of the criterion variable. Values are ordinarily between 
-1.00 and +1.00, like correlation coefficients, but may occasionally exceed these bounds. 

Linear composite : If we have a series of variables, Xj, X^, etc., we can add them 

up, with positive or negative weights (for multiplying constants) attached to them. The 
result is called a linear composite. For example, we might have reason to form the linear 
composite Y' = (+.3) X, + (-.4) X., where (+.3) and (-.4) are the weights. Then if for a 
given individual X, = 40 and X « 25, the value of Y' for that individual is (+.3) (40) + 
(—,4) (25) =2. Regression analysis can now be viewed as a process of determining the best 
weights to use in a linear composite for predicting Y from a series of variables X^, X^, 
etc. Multiple correlation is the correlation of the linear composite, so formed, with the 
criterion . 



Canonical correlation : a special type of correlation expressing the optimal relation 

between two sets of variables, Y,, Y 2 » etc., on the one hand, and X^, X^, etc., on the 
other by forming linear composites for each set and determining the optimal weights to use 
in each composite. It is the correlation between the linear composites so formed. 

We shall now turn to the analysis of certain data from the present study that we will 
use as an illustration of multiple regression analysis and canonical correlation. We 
first analyze the data by multiple regression. The canonical regression is introduced in 

section 4. 



2, Multiple Regression Prediction of MLA Skills Tes t Scores 
from MLAT and Three Background Variables. French Majors 



The complete regression analysis data for 1039 "regular" cases in French whose data 
on all the relevant variables were complete is shown in Table 8.1, which we will explain 
in detail. 



ERIC 
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Table 8.1 

Multiple Regression Analysis for Predicting MLA Skills Test Scores 
from MLAT Scores and Three Background Variables 
French Majors ^ "Regular" Cases (N * 1039) 



Criteria: MLA Skills Test 



Listening 



Speaking 



Reading 



Writing 



1 

2 

3 

4 

5 

6 



Mean 

S.D. 



Predictors 

MLAT-3 (Spelling Clues) 
MI 4 T -4 (Words in Sentences) 
MLAT-5 (Paired-Associates) 
Time Began (see coding below) 
Time Abroad (" " " ) 

Use of FL at Home ( " " ) 



.23** 

.15** 

.13** 

.. 22 ** 

.50** 

.14** 



6 

.17** 

,09** 

.05* 

-.18** 

.47** 

. 10 ** 



Multiple Correlation .59** 



. 20 ** 

,09** 

.05 

. 22 ** 

.39** 

, 20 ** 

.51** 



6 r 8 

,17** ,27** .19** 

,06* .26** .19** 

-.01 .18** .08** 

-.18** -.15** -.13** 

.37** .34** .32** 

.16** .11** .10** 



»51** 



.26** 

,28** 

.17** 

..23** 

.36** 

.13** 

.55** 



B 

.17** 

.23** 

.06* 

-. 20 ** 

.34** 

. 11 ** 



b-Welghts 



1 

2 

3 

4 

5 

6 

Intercept 



.13 

.11 

.08 

-2.49 

4.51 

2.16 

29.95 



.16 

.09 

-.02 

-3,08 

4.30 

4.19 

60.57 



.17 

.25 

.14 

-2.04 

3.40 

2.32 

28.96 



Intercorrelations of Predictors 



6 



Intercorrelations of Criteria 



1 

2 

3 

4 

5 

6 

















1 


2 


3 


4 


1.00 


.28** 


.22** 


-.07* 


.01 


-.01 












.28** 


1.00 


.32** 


.08** 


.02 


-.07* 


1 


1.00 


.69** 


.73** 


.76** 


.22** 


.32** 


1.00 


.03 


.03 


-.05 


2 


.69** 


1.00 


.58** 


.65** 


-.07* 


.08** 


.03 


1.00 - 


.06* 


-.09** 


3 


.73** 


.58** 


1.00 


.80** 


.01 


.02 


.03 


-.06* 1 


.00 


.06* 


4 


.76** 


.65** 


.80** 


1.00 


-.01 


-.07* 


-.05 


-.09** 


.06* 


1.00 


Mean 


44.14 


73.52 - 


^8.89 


47.18 


27.59 


32.33 


19.71 


2.03 1 


.74 


1.12 


S.D. 


7.59 


9.27 


8.51 


9.53 


9.81 


6.26 


4.90 


0.54 0 


.80 


0.36 














26.10 



**p < .01, *p < .05 for hypothesis value of zero. 






Coding 




Time Began 


Time Abroad 


Use of FL at Home 


1 Started in grade school 


1 Never Abroad 


1 Never 


2 Started in high school 


2 Summer or Tour Abroad 


2 Occasionally 


3 Started in college 


3 Year of Study Abroad 


3 Frequently 



The table concerns the prediction of MLA Skills test scores, one by one, from 6 pre- 
dictor variables that we know from previous chapters are relevant to foreign language at- 
tainment. In a sense, it is an assemblage of four separate ^bles, one for each of the 
MLA skills tests, because each of these is to be predicted separately from the same pre- 
dictors. The basic data from which the analysis starts are the intercorrelations of the 
predictors (shown at the bottom of the table) and the correlations of the predictors with 
each of the criteria (shown in the columns labeled "r" at the top of the table) . In the 
present case there are only a few small significant correlations among the predictors^ 
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chiefly among the MLAT tests, ^ large number of cases makes a correla- 

correlatlon with one or more of the criteria, we 1% level. (That 

tion of only .06 significant a absolute magnitude would occur by chance only 

s T, :: £ .h„.. «*, « .. 

time.) 

-•r.«n.n/s.s .ss s;: 

an intermediate stage that determines efanaardized variable is a variable 

"standardized variables” for the same purpose. ^ from the mean. 

« a e:riaMeTaI a*^ai of.Tay. 40, and a standard deviation of 5 a score of 50 

on that variable would be expressed ®® "g® i“‘*“seore of™5 0 ^ 1 ^ variable would 

standard Sandardized form. BetaJ^eights. then, are the weights for pre- 

dL:r:rriables e;pressed in standardized --i^unM^ 

relative extent b® ® lil^tLe account of the correlations among the predictor var- 

diction of f ® correlate about equally with 

lables; th.«, if t*o high^ corroiatea^p^^ assigned a high beta-weight, since 

rte rtheriariabirdois not contribute much unique information to the prediction after 
the first variable is used. 

T . n«Lf at the correlations and beta-weights (6) for the prediction of the 

Let us look, now, n^ted earlier each of the predictors has a significant 

"'""flaii^with ‘tMs crH^io^ the highest'correlation, .50. Time 

S:r®: fare :eratfvt%rrrelati» 

ciated with values of «1' ^represents starting study of the FL in grade school, 

maining predictors have positive correlations with the criterio . 

:%fi:t^;;tVt:nrp:ini,“t 

?^Hx^r tHlcr^aratJ^n^^^^^ ••®xpuln;d.. 

by the predictor variables. 

qiinilar interpretations can be derived from the figures for the prediction of Speak- 

ss;-;-;u;:rr£SC-"Sz:;.: 

scores . 

The b-weifi'hts given below the correlations and beta-weights represent the beta- 
weight tr^fllSed in^valun that are directly applicable to raw scores. For example, 
suppose we had an Individual whose scores on the predictors were as given ow. 



o 
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Individual 


Listeriing 


Score 

X 


Predictor 


A*s 


score 


b-weight 


b-weight 


MLAT-3 




35 


.13 


4.55 


MLAT-4 




27 


.11 


2.97 


MLAT-5 




21 


.08 


1.68 


Time began 




2 


-2.49 


-4.98 


Time abroad 




3 


4.51 


13.53 


Use of FL at 


home 


1 


2.16 


2.16 






•» . 


Sum 


19.91 



Intercept 2 9«95 
Predicted Listening Score 49.85 



Then we could, as the table shows, multiply j^'ach score by the corresponding b-weight, sum 
the results, and add the ^'intercept** to thf* result to obtain the best prediction of the 
Listening score. This predicted score is technically a linear composite of the predictors. 
The prediction is '‘best” in the sense that there is the smallest standard deviation of 
errors made in prediction, an "error” being the algebraic difference between the actual 
score and the predicted score. For the present data, it is only an academic exercise to 
compute predicted scores and in the future we will drop the reporting of b-welghts, since 
we are mainly interested in determining the relative extents to which criteria are ex- 
plained by the predictors, as indicated by beta-weights. 

We will, however, continue to report multiple correlations (R), since these indicate 
the overall extent to which a criterion variable can be predicted by the predictor varia- 
bles through multiple regression analysis. For the Listening criterion. Table 8.1 shows 
the multiple correlation to be .59, and it is very significantly different from zero (as 
shovm by the double asterisk). Technically, this multiple correlation is the simple 
correlation between the criterion variable and the linear composite formed by weighting 
the predictor variables by the b-weights (or by the beta-weights if the predictors are 
expressed in standardized form). That is, if we were to form a linear composite as shown 
above for each of the 1039 individuals in the sample, there would be a correlation of .59 
between these predicted Listening scores and the actual Listening scores. 

By adding more predictors, we will hope to predict the Listening scores even more 
accurately, that is, explain a greater proportion of their variance. This in fact will 
be done later in this chapter. 

Table 8.1 includes also the intercorrelations of the criteria. These intercorrela- 
tlons are not actually used in the computation of the multiple regressions since each of 
the criteria is dealt with separately. However, the relatively high in ter correlations of 
the criteria explain why the multiple regression systems for them are fairly similar. 

The only term in Table 8.1 that has not been explained is 'Intercept?'. We will only say 
that it is a number that must be included in the linear composite to make the mean pre- 
dicted score equal to the mean of the actual score. 



3. Multiple Regression Techniques Applied to Prediction 
of Each MLA Skills Test Variable, in Each Language 

An explanation of the multiple regression technique having been given, we can proceed 
to consider the results of its application to various sets of data obtained for "regular" 
cases in the several languages. Because the stability of the results over languages is of 
more interest, possibly, than the comparative results for the different skills, we will 
present results for each MLA skills test in turn. 
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Table 8.2, having to do with the predictior of Listening test scores, contains 
first column beta-weights and the multiple correlation for Listening test scores already 
f”sented^ Table 8.! for the French sample. It also contains howev« analog^sr^ 
Llts for other language groups and for different sets of variables. There are five main 

sets of results: 

(1) Data for complete samples In French, German, Russian, and Spanish, using three 
MLAT subtests and three variables that summarize the backgrounds of the students. (The 
results for French were presented in the preceding section.) 

(2) Data for complete samples in French, German, Russian, and Spanish, using J^T 
total score, amount of time abroad, the extent the FL Is used at home, and 5 variables 
that present in some detail the training histories of the student. 

(2) Data for French and Spanish samples that started study of their major foreign 
man or Russian.) 

II.\ n=i.» for French German, and Spanish samples that started study of their major 
forel^\»g»4rirMgh’sS™l.’ (lnsu?flclent dataware available for an analysis for 

the Russian group.) 

(5) Data for sallies In each of the four languages that started study of their major 
foreign language in college. 

r ru. rhroo sfits of data, the Intent was to select all applicable variables 

For ‘ md that were relatively Independent of each other. However, 

S£TC£ 

For clarity of presentation and for economy of space, slmpU correlation coefficients 
are not given. 

a j .. ahrttif cffltlstical sicnif icanco . Wlien the number of cases is re- 

latlveirsLT a^ it is for h^ R^siaS L ols, statistics such as correlations and beta- 
ifightf must ie farfer In absolute magnitude to be significantly 
flnle r»dom fluctuations « 

high m^” 24 f”l to be significantly different from zero at the 5% level • 

high as eZ4 laix tu u B j petimates we have of the values we might obtain 

ifresf'^T^e^fd.ifofpr^liior^^^^^ SsrpecS iwiisUcIf Hgnlffca^cf each 
additional variable has the same effect as dropping one case. 

fa^rianf f Upfedic^bif lltTts fa^e’^gflflfaif lfd:Uent 

FL at Home generally runs second. 

Prediction is even better when more variables are added to 

i-rr£“»-ur»KT.;'=';,ns r:;r.‘:.2s “'niX-;. 



Table 8.2 




Table 8.3 
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in terms of semesters or course hours from grade school on, is far less important than the 
kind of exposure to the foreign language that he can get at home or through study abroad. 
Language aptitude plays a small but generally significant role. 

In the case of the grade school starters, where amount of school exposure has been 
largely evened out. Time Abroad is practically the only variable that makes any signifi- 
cant difference in Listening score. Nevertheless, the overall prediction remains highly 
accurate. It is of interest that in the case of the Spanish grade school starters, the 
extent to which the instructors in college literature courses use the foreign language in 
class is a significant predictor. 

Multiple correlations between .61 and .67 are found for the prediction of Listening 
scores of students who started in high school. Language aptitude plays a small but sig- 
nificant role. Besides the major predictors (Time Abroad and Use of the FL at Home), 

Hours of College Literature is seen to be an important variable, consistently over the 

language groups. 

The pattern of results is si\’'ilar for those who started their foreign language in 
college, except that the role of the teacher's use of the language in the classroom be- 
comes even more significant. The multiple correlations remain high even when the group 
becomes relatively homogeneous in training history . 

Table 8.3 presents results for the MLA Speaking test. The overall accuracy of pre- 
diction, as reflected in the multiple correlations, is generally slightly smaller than in 
the case of the Listening test; this may be partly a function of lower reliability of the 
Speaking test criterion. Otherwise, the pattern of results is highly similar to that for 
the Listening test. 

The results in Table 8.4, for the MLA Reading test, are similar in most respects to 
those for the Listening and Speaking tests, except that language aptitude variables, es- 
pecially MLAT-4 (a test of "grammatical sensitivity") have more role than before. Also, 
the beta-weights for Hours of College Literature tend to increase slightly, as we might 
expect them to in view of the fact that college literature courses are designed to en- 
courage reading skill. Time Abroad, for some reason, fails to attain significance for the 
college starters, possibly because individuals starting in college are less likely to have 
had experience abroad (as was shown in Chapter VII) . 

Prediction of scores on the Writing test (Table 8.5) depends on much the same vari- 
ables as was found to be the case for the Reading test, with certain language aptitude 
variables slightly more emphasized. 

Somewhat surprisingly, there is only a scattering of Instances in which the beta- 
weight for Hours of College Language courses (as opposed to literature courses) attains 
significance. If anything, the trend is for these values to be negative, especially for 
students starting FL study early. This does not signify necessarily that college language 
courses are of no value. It may simply signify that individuals who arrive at college 
relatively less well prepared in language skills generally have to take more such courses 
in order to achieve the competence prerequisite for literature courses, and these are in 
general the poorer students. 



The results shown in Tables 8.2 through 8.5 indicate, in summary, that variations in 
the amounts of skill that college language majors attain in their major foreign language 
can be rather well accounted for by variations in their language aptitude and the amounts 
of exposure they have had to the language either in or out of school. 
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4. Canonical Regression Analysis of MLA Skills Test Scores 
with MLAT and Three Background Variables as Predictors. French Majors 



In this section we Introduce a somewhat more elaborate technique of statistical ana- 
lysis, canonical regression analysis, and apply it to the same data that were discussed in 
section 2, Certain purposes are attainable with canonical regression analysis that are 
not attainable with ordinary multiple regression analysis. It is applicable only when 
there are two or more criterion variables (that are not perfectly correlated) • One pur- 
pose it serves is to yield information as to what extent it is possible to predict some 
combination (linear conq>oslte) of criterion variables better than any one of them sever- 
ally can be predicted. Further, it allows ns to explore the possibility that these crit- 
eria can be meaningfully predicted in different ways. For example, if there are two crit- 
erion variables, it may be that some predictors predict the sum of these variables (or 
some sum with positive weights) , and other predictors predict the algebraic difference be- 
tween the variables (or more generally, a linear composite of the criteria such that one 
variable has a positive weight and the other variable has a negative weight) . 

Technically, a canonical correlation is a simple correlation between two linear com- 
posites, one constituted from a set of predictors, and one from a set of criteria. The 
weights for these composites are computed in such a way that they maximize the correlation 
between the composites. However, it is often the case that two or more sets of linear 
composites can be formed in such a v^ay as to maximize their intercorrelations. Such sets 
of composites are formed in a series: the first canonical correlation, the second canon- 

ical correlation, etc., each successive correlation being formed from the residual covari- 
ance . that is, the covariation that remains in the data after the effects of the previous 
canonical correlations have been removed. 

Tile weights for the linear composites are computed only in "normalized" form, i.e., 
the sum of their squares are equal to unity. Otherwise, they are analogous to beta- 
weights, except that they are computed in two sets for any given canonical correlation — 
one for the predictor variables and one for the criterion variables. The statistical 
significance of the canonical correlations can be tested, but of course the same con- 
straints concerning the number of cases and the number of variables apply. As the number 
of cases decreases and the number of variables increases, the canonical correlation must 
be ever larger in absolute magnitude to attain statistical significance. No statistical 
tests of canonical regression weights were available in the computer programs we used. 

We may now examine Table 8.6, which presents a canonical regression analysis for the 
data of Table 8.1. The intercorrelations of the criterion variables as well as those of " 
the predictor variables are used in canonical regression analysis, but to save space they 
are not repeated in Table 8.6. It was found that four canonical correlations were sig- 
nificant, although the size of the last two is very small. It is a characteristic of 
statistical data that very small effects can be significant when there are very large 
numbers of cases, as in the present instance. 

The first canonical correlation system, with a coefficient of .61, obviously refers 
to the overall predictability of the criterion tests, with positive weights, from the pre- 
dictor variables weighted as indicated. The regression coefficients for the predictors 
(listed at the left of the table) are in the column headed P, while those for the criteria 
(as listed at the right of the table) are in the column headed C. The Listening test 
turns out to be the most predictable of the criteria; this finding agrees with that of the 
multiple regression analysis. Thus, the best overall measure of French language compet- 
ence, from these results, would be one in which standardized scores on Listening, Speak- 
ing, Reading, and Writing are added up with weights of .87, .33, .03, and .37 respective- 
ly. 

The second canonical can be interpreted as representing a tendency for the superior- 
ity of a student in Reading and Writing (positive weights) over his ability in Listening 
and Speaking (negative weights) to be predicted chiefly by MLAT-4 (Words in Sentences), 
and secondarily by the amount of time he has not spent abroad (since Time Abroad has a 
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Canonical Regression Analysis for Data of Table 8.1 
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negative weight on the predictor side). This finding has much logical appeal: MlJ^T-4 

measures "grammatical sensitivity," which might be expected to enhance a student's per- 
formance In Reading and Writing, particularly the latter, calling as It does on high com- 
petence In the grammatical aspects of the language; and students who have not been abroad 
might be expected to devote themselves more to the written aspects of a language or at 
least to have achieved relatively less competence In spoken aspects. 

The third canonical, with a relatively small coefficient of .15, appears to refer to 
a prediction of the superiority of active performances In the language (Speaking and Writ- 
ing having positive weights) over passive performances (represented by Listening and Read- 
ing) . This kind of superiority Is associated with those who began French study early 
rather than late, who have homes where the foreign language Is used occasionally or even 
frequently, who are poor on rote memory, but who have not spent much time abroad. The 
negative weight of time abroad Is a trifle puzzling In view of the pattern of other co- 
efficients, for one would expect students who have spent much time abroad to be superior 
In active skills. But then, this canonical applies only after the effects of the first 
two have been extracted, and Time Abroad was already shown to be associated with Listening 
and Speaking competence. 

The fourth canonical, though significant. Is difficult to Interpret. It seems to 
show that after all the other covariation Is taken account of, there Is a tendency for 
students with high scores on MLAT-3 and who began French late show superiority In Reading 
and (to some extent) Speaking, as contrasted with Listening and Writing. 

5. Canonical Regression Techniques Applied to 
Prediction of Linear Composites of MLA Skills Test Scores 

Canonical regression techniques were applied to the same samples of students, with 
the same sets of variables, as the multiple regression techniques. Each canonical re- 
gression system contained four canonical correlations, varying numbers of which were 
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statlstlcally significant, depending largely on the numbers of cases involved* The char- 
acter of a linear composite determined by a canonical correlation is not necessarily as- 
sociated with the order (first, second, third, or fourth) in which it is extracted from a 
given set of data* Therefore it seemed wise to examine all the data in terms of the pat- 
terns of coefficients for the criterion variables, sorting the canonical composites on the 
criterion side into roughly similar patterns* With allowances made for sampling fluctu- 
ations arising from limited numbers of cases in some instances, it was possible to estab- 
lish four such patterns, each named in terms of the combination of MLA skills tests it 
represented : 

(1) A general foreign language competence factor * These were criterion composites 
for which all or nearly all the coefficients were positive (or at least unidirectional); 
in most cases two or three of the coefficients were f Airly substantial in magnitude* In 
almost all Instances these composites were those yielded by the first canonical correla- 
tion* Sixteen of the 17 canonical correlations were statistically significant at the 5% 
level or better* These composites were therefore Interpreted as representing a general 
competence factor in the relevant language, inasmuch as they set forth the way in which 
the criterion variables could be optimally weighted to be predicted by a set of variables 
known to be unlvocal in predicting the several measures of competence* This general for- 
eign language competence factor corresponds to the general factor found in the Intercorre- 
lations of the MliA skills test scores in Chapter IV* The canonical regression coeffici- 
ents for the several sets of data analyzed are to be found in the lower four rows of Table 
8*7* It will be seen that the coefficients for the Listening test are generally the high- 
est, agreeing with previous findings that the Listening test is the best overall measure 
of skill in the foreign language* 

(2) A factor representing superiority in written over spoken skills * This will be 

termed the WR factor, denoting the fact that the coefficients are usually positive for the 
Writing and Reading tests and negative for the other two tests* These coefficients are 
shown in the last four rows of Table 8*8* The directionality chosen was largely arbit- 
rary: the factor could equally well have been termed the LS (Listening and Speaking) 

factor, with the coefficients reversed in sign, except that it seemed desirable to retain 
a positive direction for certain Important predictors* (The directionality of a linear 
composite is arbitrary, except that when the criterion coefficients are reversed in sign, 
the coefficients for the predictors must also be reflected* Reflections of sign were made 
for both predictors and criteria so that as much agreement as possible was obtained among 
the canonical composites obtained for the several sets of data*) This factor may be re- 
garded as a measure of the extent to which a student's skills in Reading and Writing, on 
the one hand, are different from his skills in Listening and Speaking* With the direct- 
ionality chosen for the coefficients, it is a measure of the student's superiority in 
written over spoken skills* In most Instances, this factor was associated with the second 
canonical correlation, indicating that it was the second most Important source of variance 
in the data* Twelve of the 17 canonical correlations were statistically significant at 
the 5% level or better* 

(3) A factor representing superiority in productive skills * This will be termed the 

SW factor, reflecting its largely positive coefficients for Speaking and Writing as con- 
trasted to laTiTgely negative coefficients for Listening and Reading* The coefficients are 
shown in the last four rows of Table 8*9* As in the case of the second factor, the di- 
rectionality chosen was arbitrary* With signs reversed, it could equally well be inter- 
preted as a factor representing superiority in receptive skills* It was associated about 
equally with the third and fourth canonical correlations, and only five out of 17 of these 
canonical correlations were significant at the 5% level or better* Thus, this dimension 
is a relatively weak source of variance* But it nevertheless makes some psychological 
sense * 0 

(4) A factor representing superiority in reading and speaking * This is here termed 
the RS factor* The coefficients for the criterion variables are shown in the last four 
rows of Table 8*10* This factor was a weak source of variance in the data, as all the 
canonical correlations were relatively small in magnitude, and only four out of 17 of 
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them were statistically significant at the 5% level or better, ^2 %eL of 

purely chance level. There are, in fact, only 3 ways of dividing ^things into 2 sets of 
2 eS ^d this factor may represent little more than a chance residual from other sour- 
ces of’variance. It is a little hard to imagine just what superiority in reading 
speaking (as contrasted to listening and writing) would imply. 



Let us now consider the results shown in Table 8.7. The canonical regression weights 
for particular combinations of predictors are shown in the first part of the table, fol- 
lowed by the values of the canonical correlations, probabilities (given only where they 
are greater than .05), and numbers of cases (N*s). The data of this table generally match 
those of Tables 8.2 through 8.5 because this table, like those, has to do with measures 
of overall competence in the relevant foreign language. The almost 

dictor of measured competence is the Amount of Time Spent Abroad, and this agrees 

with those noted elsewhere in this study. The results shown here emphasize 
Time Abroad is particularly associated with Listening, and also (though to a lesser ex- 
tent) with Speaking. 

When three separate MLAT sub tests are included in the analysis, tests 3 and 4 are 
more likely to show positive coefficients for predicting overall language coi^etence. In 
the second set of data (under the heading "Complete samples. ^T, Background, and Hist- 
ory") the MLAT Total shows all positive and sometimes' substantial coefficients. 

Time of Beginning FL Study shows negative coefficients that reflect the fact that 
those who started earlier are likely to achieve higher skill than those who start late. 

The coefficients for Use of the FL at Home ate in the expected direction and ate usu- 
ally of substantial magnitude, showing that this variable is ^ iii^ortant iniluenc , 
dependent of the other variables, in the attainment of overall skill. 

The data for various indices of length and quality of exposure to formal schooling 
in the foreign language are generally such as to suggest that this formal training is in- 
flu^I^tiil In^enhanllnl forel|n language skill. The coefficients for the indices relating 
ti conege Ut“^^ are oftin sizable, although they are never ^ large ^ those 

for Time* Abroad. Only in the college-starting group in Spanish do we find a sizable co- 
nfident for the influence of language competence of the teacher in * 3 

course. Some of the coefficients for language courses are negative, especially for groups 
who should have been able to complete introductory language courses at an earlier stage. 

Table 8 8 it will be recalled, has to do wiih a dimension of language competence 
whereof tn inindLl shows significantly different skills in reading 

posed tb his skills in speaking and listening. We have chosen the direction of 'bi 

polar" scale so that positive coefficients imply a superiority of reading and writing ov r 
?he listening and spewing skills The reason for choosing this direction will now be 
evident* the best predictor of this dimension is generally subtest 4 of the M^T (or, 
whe^rt^at subt^st Ls not been separate!, entered into the analysis, ^he total score from 

a ^AT e..hrp^-Q’> Subtest 4 of the MLAT, called Words in Sentences , is chiefly a 

sensitivity," i.e., his abliHy to identify and 

relate the grammatical functions of words in sentences. It is not surprising that such an 
flbilitv mieht be associated with the individual’s superiority in the written aspects of 
fls f^refgf U^rge at^Lnments. particularly as n«asured by the Writing ‘ of the 

MLA battery, since the Writing test probably puts the student on his grammatical mettle 
more than any other of the MLA skills tests. 

There does exist, however, an apparent anomaly in these results, in that ^ JJf® 
desiened and validated to predict rate of learning in audiolingual skills, ^y 
riooQ it now show better prediction of reading and writing skills? In the first place, it 

a. rviflf- onlv the Words in Sentences subtest (MLAT=*4) shows the generally ano- 

ZL- roButts- freque^y thr^^^^ negative coefficients for the writing 

“J«ioZ; faltorrfaid tSus, in effect, are positively associated with audiollngual 
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superiority) . Secondly, the students contained in our samples are presumably o^s ^ 
have had, in many cases, many years of exposure to foreign language training. The 
was iesigned to measure mainly rate of learning in introductory language training. The 
students in our samples have generally long since passed the stage of introductory lang- 
uage training and have had an opportunity to overcome any effects of deficient general 
language aptitude; that is, those of lower aptitude, while taking perhaps than 

thoL of higher aptitude, have nevertheless had sufficient time to achieve minimal compet- 
ence in their major language. 

A variable that has generally negative coefficients in predicting superiority of 
reading and writing over audiolingual skills is Amount of Time Abroad. These values could 
indicate simply that Time Abroad is positively associated with superiority in listening 
and speaking skills; it is also possible, however, that those who stay at home would 
have more opportunity to cultivate their reading and writing skills. 

The findings for Use of the FL at Home indicate, similarly, that exposure to the for- 
eign language at home is associated with superiority of audiolingual skills . 

The results for the several variables reflecting length and quality of formal foreign 
language training are none too consistent over the various samples and language groups. 
Variables having to do with college literature courses do. however, generally have posi- 
tive coefficients suggesting that such courses promote reading and writing skills more 
than they do audiolingual skills. Conversely, variables derived from e^eriences in- 
troductory and intermediate language courses tend to have negative coefficients, sugg 

promote audiolingLl skills. The patterns of coefficients across languages 
suggest that French and Spanish language courses tend to concentrate on audiolingual 
skills while German and Russian language courses emphasize the promotion of r®n^ing and 
writing skills. Nevertheless, even in German and Russian, the results suggest that stud- 
ents with experience in classes where they are required to use the foreign language tend 
to gain in audiolingual skills more than in reading and writing skills. 

A possible dimension of superiority in productive skills (speaking and writing) » the 
SW factor, is dealt with by Table 8.9. None of the canonical correlations are as high as 
those generally found in Table 8.7 or 8.8, and the patterns of canonical regression 

weights for the predictors are not very consistent across samples or lanpages. The la g 

uage aptitude subtests show a very puzzling pattern of results: Lsi- 

positivc coefficients, MLAT-5 generally negative coefficients, and MLAT-3 someti^ p 
tive and sometimes negative coefficients. It does not seem worthwhile to speculate about 

these differences. 

The coefficients for Time of Beginning suggest that in French and Spanish, those who 
beein earlv are *e likely to attain superiority in productive skills, while in German 

and Russian early beginning is more often associated with superiority in recept ve s s. 

These differences may reflect differences in the ways in which these languages are typic- 
ally taught, German and Russian more often being taught with a reading comprehension o - 
jective than with a production mastery objective. 

Somewhat surprisingly. Amount of Time Abroad is more associated with superiority in 
receptive than in productive skills. At the levels of mastery typically attained by 
studLts in our sample. Time Abroad may have more effect in enriching the receptive ex- 
perience of the students than in promoting active language skills. 

In contrast. Use of the FL at Home is more likely to be positivel y, associated with 
the development of productive skills, according to the data. Students with exposure to 
the foreign language at home are perhaps more likely to be at the level of 
Sulsitl for iLl to profit from tMs experience in the direction of developing productive 

skills. 

Tae coefficients for variables derived from data on the formal training of the stud- 
ents suLeH that introductory and intermediate courses tend to promote product!^ skills, 
while literature courses promote receptive skills. These findings square w xp 
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tlons. It Is noteworthy, however, that £or high school starters, active use of the lang- 
uage by students In college literature courses promotes receptive skills, while for col- 
lege starters. It promotes productive skills. 

It Is difficult to interpret the results for the RS factor, shown in Table 8,10. 

Even If we restrict attention to the few canonical correlations that are significant at 
the 1% level, the patten^s of coefficients for the predictors are highly Inconsistent. 
Perhaps Table 8.10 Is best left as an exhibit on which the reader can test his imagina- 
tion. 



6, The stability of regression systems over institution size 

strata and institution types 

In Chapter V, It was shown that there were large differences among strata of insti- 
tution-sizes and between institution types (public and private) In variables having to do 
with student backgrounds and with Instructional procedures. There was concern over wheth- 
er institutional differences In mean MLA skills test scores might be largely due to such 
differences. Even though It was shown that the mean MLA skills test score differences re- 
mained even after certain statistical controls were applied, there was still some doubt as 
to whether problems of Institutional differences had been adequately resolved. 

A further attempt was made to resolve these problems by applying the canonical cor- 
relation technique within strata and institution types. It was reasoned that if the first 
canonical correlations were found to be drastically lower In such analyses than they were 
in tabulations done across strata and institution types, the conclusion could be drawn 
that identifiable stratum and institution-type differences were largely responsible for 
the variation in mean MLA skills test scores. This conclusion would tend to cast doubt 
on the success of the sampling design In drawing a representative sample of institutions. 
If, on the other hand, the canonical correlations remained high, one would conclude that 
sufficient variation existed within cells of the sampling design to permit one to dis- 
count between-cell differences as being Important sources of variance. 

Table 8.11 presents the results of this analysis. It was performed only for French 
because this was the only language with enough cases to permit sub -classifications with 
substantial numbers of cases. (Even so, students In Public Stratum 1 were not numerous 
enough to justify application of the canorulcal correlation technique.) The table pre- 
sents data only for the first canonical correlation. As may be seen In the table, the 
canonical correlations were fully as high as, if not higher than, those obtained for the 
total sample In French. In general, the patterns of large coefficients are similar across 
strata and institution types. Time Abroad Is nearly always the best predictor; the ex- 
ception Is In Public Stratum 4, and this is possibly because the criterion composite seems 
to represent not overall competence but the SW factor (the contrast between Speaking and 
Writing, on the one hand, and receptive skills of Listening and Reading). In any case, 
the correlation is barely significant. Time Began is generally a negative predictor, as 
it has been found to be for the French sample as a whole. Hours of College Literature, 
or the Instructor's use of the FL In college literature courses, is a reasonably consist- 
ent predictor. 

d> 

It may be concluded, we feel, that wlthln-cell variations In student background and 
experiences are of sufficient Importance to allow us to discount the Importance of be- 
tween-cell differences in accounting for the results of this study. 
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Chapter IX 

CHARACTERISTICS OF MODERN FOREIGN LANGUAGE DEPARTMENTS 
IN RELATION TO STUDENT ACHIEVEMENT 



1. Introduction 



Chapter V showed that certain characteristics of institutions — namely j, institution 
size and type (public vs. private) — ^were related to student achievement. Its findings 
left the impression that student achievement might be related to variations in instruct- 
ional policies and procedures. 



Besides data collected from the students themselves concerning the characteristics 
of the instruction they had had, information was obtained by means of the Quest ionna _ ir e 
for Chair men of Foreign Language Departments (Appendix C) , completed and returned by at 
least one of the foreign language departments in 198 of the institutions participating in 
the national study. In all, 334 questionnaires were returned. The number of question- 
naires returned from a given institution depended partly on the number of separate foreign 
language departments at the institution and partly on the willingness of the department 
chairmen to complete and return the questionnaire. According to our records, the 334 
questionnaires returned represent 91.0% cf the total number, 367, of separate departments 
in the 202 participating institutions. This is a high rate of return. 



The analysis of the data contained in these questionnaires has been reported in. a 
doctoral thesis by Fannie A. Handrick (Handrick, 1967). Data from both the pijot study 
sample in the state of New York and the national sample were studied. Since this thesis 
is (or will be) available both through interlibrary loan and also through microfilming 
services (including the ERIC system of the U. S. Office of Education), we present here 
only a summary of the major findings; we restrict this summary to data from the national 

sample. 



The unit of analysis was sometimes the "department" and sometimes the "subdepart- 
ment " defined as a department or part thereof devoted to the teaching of one 
languages included in this study. In this sense, the 334 departments represent 769 sub- 
departments. Table 9.1 shows a classification of these subdepartments by department type, ^ 
institution size, and institution type. "Department type" is indicated either as single, 
"family " or "all." "Single" means that the department was devoted to only one language 
(e.e. Department of French); "family" means that the department was devoted to a farai y o 
languages (e.g. Department of Romance Languages); "all" means that the department embraced 
all the foreign language teaching (at least, modem languages) that was done at the in- 
stitution. As one might expect, subdepartments responsible for instruction in a single 
language are most often found in large, private institutions. 



The assumption was made that the instructional policies and procedures reported by 
the department chairman (or whoever filled out the questionnaire) applied to « all of 
the lAuage instruction covered by the department. For example, when the subdepartm^t 
was the unit of analysis, the data from a questionnaire returned by a department of Rom 
LL^gefwS U^ed icr the analysis of both the French and the Spanish instruction. 



2. Dimensions of Instruction in Subdenartments 

Factor analytic techniques were applied to the questionnaire data to identify the 
chief dimensions in which instructional policies and procedures in beginning and inter- 
mediate courses varied. There were four such dimensions identified: 

I. The extent to which the use of English is avoided in first year courses (i.e., 
introductory language courses) . This dimension reflected chairmen s responses to a ques- 
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Table 9.1 



National Study: Classification of Sub-Departments by Type 

of Department and Type and Size of Institution 





French 
N » 189 


German 
N « 178 


Italian 
N » 64 


Russian 
N = 129 


Spanish 
N « 189 


Total 
N = 769 


Department Type 
Single 
Family 
All 


33 
' 40 
116 


37 

28 

113 


8 

37 

39 


12 

17 

100 


32 

40 

117 


122 

162 

485 


Institution Size 










25 

28 


81 

103 


1 (small) 

2 


25 

28 


0 "20 
25 


4 

5 


7 

17 


3 


60 


5 


18 


36 


58 


227 


4 


40 


39 


25 


34 


41 


179 


*T 

5 (large) 


36 


39 


32 


35 


37 


179 


Institution Type 
Private 


121 


110 


44 


69 


121 


465 


Public 


68 


68 


40 


60 


68 


304 



tion that asked haw often English was used to explain points of grammar or the meanihgs 
of new words and phrases. The major contrast was between those departments that reported 
English was used "frequently" for these purposes and those departments reporting occas- 
ional" use of English. 

II. The extent to which the use of English is avoided in second-year (fntemediate) 
courses. Here the major contrast was between departments reporting "occasional use of 
Enelish and "rare" use of English. There was some tendency for these two dimensions to 
correlate; that is, departments reporting "occasional" use of English in the fi^st year 
were more likely to be those departments reporting "rare" use of English in the second 
year; likewise, departments reporting "frequent" use of English in the first year were 
more likely to be departments reporting "occasional" use in the second year. 

III. The emphasis given to audiolingual objectives. The major contrast was between 
those departments reporting "more emphasis on listening and speaking and those depart- 
ments reporting "equal emphasis given to all four skills." (Very few departments reported 
that they gave more emphasis to reading and writing than to listening and speaking.) 

IV. The opportunity given for voluntary use of a language laboratory. The major con- 
trast was between departments requirii^ use of the language laboratory m basic skills 
courses and those departments allowing purely voluntary use of the language laboratory. 

Factor scores based on these dimensions were assigned to 656 subdepartments with re- 
quisite data, with a view to relating these departmental characteristics to data on stud- 
ent achievement. 

There was some question, however, as to whether the data based on the reports of 
department chairmen corresponded to student reports concerning what went on in the class- 
roL. Information was assembled from the reports of students who started study of their 
major foreign language at the college level. Ratings of first and second year courses 
listed by these students in the Questionnaire for F oreign Language Ma j o^ (Appendix B) 
were averaged within subdepartments . By using data only from subdepartments where there 
were at least two students contributing ratings, it was possible to estimate the average 
reliability of these ratings. The reliabilities so estimated are as follows: 



o 
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Reliability 



Teacher's use of English, first 
year courses 

Teacher's use of English, second 
year courses 

Audiolingual emphasis 

Use of language laboratory . . 



.39 



.63 



.66 



.67 



When these ratings were compared with the factor scores assigned to the departments, the 
correlations were significant but very low, as shown by Table 9.2. 



Table 9.2 



National Study: Correlations Between Four Factors of 

Audiolinguality and Mean Student Ratings of 
Classroom Practices 



Factor 


Student Rating 


N 


r 


First year classroom 
use of English 


Teacher ' s language , 
first year 


245 


.19** 


Second year classroom 
use of English 


Teacher ' s language , 
second year 


244 


.20** 


Audiolingual 

emphasis 


Audiolingual 

emphasis 


243 


.13* 


Opportunity for voluntary use of 
language laboratory 


Use of language 
laboratory 


243 


-.06 



**p < .01 

*p < .05 



It should be noted, however, that the questions asked of the students and of the depart- 
ment chairmen were not precisely comparable. Highest agreement is found when the ques- 
tions were essentially the same, viz . to what extent does the teacher speak English in the 
classroom? The questions concerning audiolingual emphasis and use of the language labor- 
atory asked of students and of department chairmen were less similar. Nevertheless, the 
overall amount of agreement between students' and department chairmen's ratings was not 
great enough to justify thinking that the two sets of ratings referred to the same things, 
and this fact must be borne in mind as the data are examined. 

Subdepartments were also scored for "advanced instruction variables": number of 

degree requirements, number of required and voluntary activities offered (opportunities 
for study and travel abroad, language houses, clubs, a pledge to speak only the foreign 
language at designated places and times) , and numbers and types of courses offered. A 
summary of the data bearing on advanced instruction is given in Table 9.3. These data may 
be summarized conveniently by ^e statement that subdepartments teaching the more popular 
languages (French, Spanish, Geraan) have more degree requirements, offer more associated 
language-learning activities, and offer more courses than subdepartments concerned with 
the less popular languages (Italian, Russian). These advanced instruction variables are 
also related to institution size: larger institutions tend to score higher on them than 

smaller ones do. Furthermore, public and private institutions differ in these advanced 
instruction variables: publib institutions, while offering more types of courses, tend to 

report fewer degree requirements and associated language-learning activities than private 
institutions do. 
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Table 9.3 

National Study: Advanced Instruction Variables 



A. Percentages of Sub -departments Mentioning Each Type of Degree Requirement 





French 


Spanish 


Italian 


German 


Russian 


Requirements : 


N “ 189 


N « 189 


N = 84 


N = 178 


N » 129 


Advanced Skills 


85.8 


81.1 


27.4 


70.6 


45.4 


Literature 


78.0 


72.8 


27.4 


62.7 


41.6 


Culture & Civ. 


36.9 


32.8 


7.1 


28.0 


16.9 


Linguistics 


8.8 


9.9 


3.6 


9.5 


6.2 


Written test 


16.6 


16.6 


2.4 


10.6 


3.9 


Oral test 


3.6 


4.2 


1.2 


3.4 


.8 


Percentages of Sub-departments Requiring Various 


Types of 


Activities 






French 


Spanish 


Italian 


German 


Russian 


Activities: 


N - 189 


N » 189 


N = 84 


N = 178 


N - 129 


Year abroad 


— 


— 


— 


0.6 


— 


Summer abroad 
Language house 


0.5 


0.5 




0.6 


“ 


(school year) 
Language house 


1.6 


1.0 




0.6 




(summer) 






— — 


— — 




Language table 


2.1 


2.6 


— 


1.7 


2.3 


Language club 


5.0 


3.6 


— 


3.4 


' 0. 8 


Pledge 


1.0 


1.0 


— — 


0.6 


1.5 


Percentages of Sub-departments Offering Various 


Activities on a Voluntary Basis 




French 


Spanish 


Italian 


German 


Russian 


Activities: 


N = 189 


N = 189 


N = 84 


N = 178 


N = 129 


Year abroad 


56.7 


49.4 


26.2 


48.2 


10.0 


Rummer abroad 
Language house 


53.6 


53.6 


21.4 


43.1 


19.3 


(school year) 
Language house 


15.1 


8.3 




7.3 


4.6 


(summer) 


5.7 


3.6 


1.0 


3.4 


3.9 


Language table 


41.6 


33.8 


10.7 


40.9 


20.8 


Language club 


89.2 


70.2 


42.8 


63.8 


46.2 


Pledge 


5.0 


5.0 




4.5 


6.9 


Percentages of Sub-departments Offering Various Types of 


Courses : 






French 


Spanish 


Italian 


German 


Russian 


Types of Courses: 


N = 189 


N = .189 


N = 84 


N = 178 


N = 129 


Beglt..ilng 


93.6 


95.2 


95.2 


96.8 


96.3 


Intermediate 


96.2 


94.6 


84.5 


96.3 


92.4 


Composition 


88.9 


86.3 


38.1 


80.1 


53.9 


Conversation 


86.3 


85.3 


32.1 


80.1 


54.7 


Phonetics 


— 


— 




— 


— 


Advanced Grammar 


54.6 


52.0 • 


13.1 


36.4 


29.3 


Literature 


95.2 


90.5 


55.9 


90.2 


64.7 


Special Reading 


36.9 


26.0 


13.1 


57.1 


42.4 


Culture & Civ. 
Diachronic 


61.9 


56.2 


15.5 


47.0 


29.3 


Linguistics 

Synchronic 


21.3 


18.7 


14.3 


25.8 


20.0 


Linguistics 


25.0 


25.0 


11.9 


23.0 


23.1 
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3. Departmerttal characteristics and student achievement 

Complete data on departmental characteristics, mean student aptitude, and mean stud- 
ent achievement were available for 163 of the 202 institutions participating in the nati- 
onal study. These institutions represented 383 sub-departments of foreign languages 
apportioned among the five languages as follows: French, 133; German, 86; Itall^an, 10; 

Russian, 28; and Spanish, 126. Because of the small numbers of subdepartments in Italian 
and Russian, data are not analysed for these languages. 

In computing mean student aptitude (from the MLAT total score) and mean student 
achievement (from the four MLA skills tests), only data^^from "regular" cases were used 
(see Chapter III). Two sets of subdepartment scores were computed. The first was derived 
from the mean scores of students who started their major foreign language study in college; 
the second was derived from the mean scores of all students majoring In the subdepartment, 
regardless of when they started their foreign language study. The former scores were de- 
rived because It was reasoned that data on subdepartment policies for beginning and inter- 
mediate language courses would be relevant only for students who would have taken such 
courses, namely, those starting In language at the college level. The latter set of 
scores, however, would be relevant for all students majoring In the subdepartment, because 
they could conceivably have been affected by policies and procedures in advanced language 
instruction. The means were computed over the stude^ ,s actually in a subdepartment, even 
If the basis for such a mean was only one student, as was sometimes the case. The numbers 
of students involved in each set of scores, over all subdepartments, are shown in Table 
9.4. 



Table 9.4 



Numbers of Students Tested in Each of Two Groups by Language 



Language 


Started in 
College 


All 

Students 


French 


191 


1140 


German 


174 


340 


Italian 


14 


25 


Russian 


73 


97 


Spanish 


167 


833 




619 


2435 



Unfortunately, the reliability of mean scores computed in this way tended to be low, 
particularly for scores based on students starting in college. To estimate the reli- 
ability coefficients for such mean scores, the Intraclass correlation was obtained (Ebel, 
1951) for means based on 2 or more students within a subdepartment: when these reli- 

abilities were found to be low, the computations were restricted to subdepartments having 
at least 3, or in some cases 4, students, but such restriction reduced the number of sub- 
departments drastically. 

9 

For students starting their major language in college, only in Spanish was it found 
that mean aptitude aiid MLA skills test scores were sufficiently reliable to constitute 
variables that could be used in studies of the relation between departmental variables and 
mean student aptitude and language achievement. , The intraclass correlations based on sub- 
departments with 2 or more college-starting .students ranged from .92 for Writing to .96 
for Listening and for Speaking, in the case of the MLA skills tests; the intraclass cor- 
relation for mean student aptitude was .78. (High reliability signifies, essentially, 
that variation between subdepartments was significantly greater than it was within sub- 
departments. ' ^ , 
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in The%u^ose was to see whether departmental characteristics,^ 

M^eported either '’J ^SSlirteLs*!^ IrtMUy.^ln^Uuuin'siae w«” 

thr<^rp^dlctor*whosrbeta-weights were more or less consistentl^y of^in- 

«uiuoM\rnnnot*rSy“irthe o&L^be^^^ghM . There is a 

28 04 .23, and .24 for the four MLA skills tests respectively, a correlation as larg 

■^®'o4 hiin. Olmlficant at the 52 level.) None of the other factor scores for depart- 

** t- 1 ^characteristics related to beginning and intermediate language instruction showed 

4«4 ant trends except that if anything, departments reporting emphasizing listening 
slgniflcmt trends, exrept that « achievement in their students than 

md ^ equally. Among the variables derived 

fX%“dent reports! those related to the ‘^“her's ®"8llsh 

yew classes avoided the use of English tended to be departments that had soM- 

whw swlrlw student mean scores on the MLA skills test; S* 

lations were .27, .18, .45, and .36 for Ustening, Speaking, Beading and Writing, 

respectively. 

The validity coefficients of the student ratings are, in general, 
nf the factor scores derived from department chairmen's questionnaire responses. Appar 
studwtra^nr. betre! positloS to know what goes on in classro^ than ^^rt^nt 
chairmen are. On the other hand, the effects reported here are very small in magnitud , 
undoubtedly they are overridden by more Important variables such as the amount of time 
abroad and others that we have examined in previous chapters. 

Handrick has also reported regression systems based on data from all students within 
a department regardless of their tl^ ‘’^i"*rent®ln‘SS skiUs reits wHuf sibdfpwreents 
Se iHenerel quite rlliable, showing that departments 

e™: reared forl33“= ruS^a??::n:! if d?irbe- 

ment typ'/ is generally the only significant predictor of mean student ^ s 

^ That -fc to sav single-language departments, and departments in larger ins 

s™ ,.S". ;sb:s*;us!n. ««■«;- 

nss;.2:.;“s: - 

-.as r/sr- 

and in Spanish it is sipiricantxy r . policy specified in terms of avoidance 

Writing. One wonders, however, how a departmentax pox cy f otnrfents most of 

i in first-vear courses could be related to mean performance of students most or 

In flr^t-vear courses reflects a general departmental policy that also ^ 

• iL mf nd"nc:d courH;. Unfortunately, the department 'halr-n were not wked 

ft^f Sgtt htre titainfd twSer^confirnatlon of findings in earlier chapters that sug- 
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Table 9.5 

Multiple Regression Data for Prediction of Me an 
Student Achievement from Selected Departmental Variables 
N =* 70 Spanish Subdepartments, 

Variables derived from Students who Started 
Study of Spanish In College 
(from Handrick, 1967) 



Variable 


Listening 


MLA Skills Test 
Speaking Reading 


Writing * 


r 


6 


r 


3 


r 


3 • 


r 


'3 


Mean MLAT 


-.09 


0 

• 

1 


-.15 ■ 


-.07 


.07 

j 


.07 


-.06 


-.01 • 


Department Type^ 


-.31* 


-.17 


-.03 


.08 


1 

• 

1 


.21 


-.33* 


-.22 


Institution Slze^ 


.36** 


.36* 


.20 


.30* 


.35* 


.29* 


.39* 


.19 


Institution Type^ 


-.11 


-.29* 


-.03 • 


-.12 


.07 


.06 


.14 


.06 


Department Chairmen's Data: 


















Avoid English (1st year courses) 


.28* 


.18 


.04 


.03 


.23* 


.09 


.24* 




Avoid English (2nd year courses) 


.04 


.02 


-.04 ■ 


-.07 


.16 


.19 


.03 


.04 


Audlollngual Emphasis 


-.05 


-.14 


-.19 ■ 


-.15 


-.01 - 


.04 


-*03 


-.11 


Opportunities for volunteer lab, use. 


-.03 


-.06 


-.06 - 


-.15 


.15 


.13 


. -.03 


-.12 


Degree Requirements 


.01 


-.07 


-.04 


.04 


-.02” - 


.24* 


-m07 


-.11 . 


No, of Activities 


-.04 


-.09 


-.08 - 


-.12 


-.01 


.09 


-.14 


-.14 


Types of courses offered 


.10 


.06 


-.08 - 


-.12 


-.06 - 


.18 


. 12 


.02 


Mean Student Ratings: 


















Avoid English (1st year courses) 


.14 


.15 


1 

• 

1 


-.19 


.24* 


.20 


.12 


-.04 


Avoid English (2nd year courses) 


.27* 


.15 


.18 


.18 


.45* 


.28* 


.36* 


,22 


Audlollngual Emphasis 


.12 


-.10 


.04 


.01 


.27* - 


.12 


.25* 


-.15 


Language lab , use 


-.07 


-.12 


.09 


.03 


• 

0 

1 


.02 


-.05 

# 


-.18 


Multiple R (shrunken) 


.35 




(.00) 


.43 




.37 



**p < .01; *p < ,05 (two-tailed test) 

‘Coding: Department Type: 1, Single language; 2, Family of languages; 3, All lang- 

uages 

Institution Si; 2 e: 1, Small; . . .,5, Large 

Institution Type: 1, Public; 2, Private 
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gested that emphasis on speaking the foreign language In advanced language and literature 
courses was beneficial In enhancing language skills of the students. 

There was no support In these analyses for the notion that use of a language labor- 
atory makes any critical difference In student performance. We have already noted that It 
Is only the smaller Institutions, that may not have sufficiently competent teachers, that 
seem to emphasize the use of the language laboratory at the college level. This is not to 
say that the use of a language laboratory in larger Institutions is pointless, but as far 
as our results are concerned, it does not seem to make any special contribution to improv- 
ing student performance over and above what may be contributed by teachers who are compet- 
ent in the language they are teaching and who emphasize its use in the classroom. 

On the whole, these analyses of the relation of departmental characteristics to 
student performance leave many questions unanswered. It is probable that the Question- 
naire for Foreign Language Department Chairmen failed to ask the right questions about 
instructional practices and policies— —questions that would Illuminate the differences 
between "good" departments and "poor" departments. The fact that our analyses shoved 
reliable differences among departments In mean performance of their students presents a 
challenge to future researchers. 
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Chapter X 



RESULTS FROM THE MLA TEACHER PREPARATION TESTS 



lo The Sample 

Coordinators at participating Institutions were told , (Appendix D-4^ : "The COl^LETE 

Battery of 7 tests [constituting the MLA Foreign Language Proficiency Tests for Teachers 
and Advanced Students] may be administered only to those senior class students who are en- 
rolled at your Institution In a program which Is expressly oriented toward the preparation 
of elementary or secondary school language teachers, and which usually Involves the com- 
pletion of a certain series of courses necessary for state certification purposes." 

Of the 2874 students who took any of the MLA tests, 1116 (38.8%) took one or more of 
the Teacher Preparation sub tests, tests 5, 6, and 7, and may therefore be presumed to have 
been enrolled In teacher certification programs. No Information has been developed as to 
whether those enrolled In such programs were more likely than othe.rs to volunteer to par- 
ticipate In the testing program, or whether, therefore, they biased the sample to any ex- 
tent. Students enrolled In teacher preparation programs may have had a special Incentive 
to participate since they might have expected to have to take the MLA battery at some 
future time as a part of teacher certification procedures or as an examination to qualify 
for a teaching position; the opportunity to take the MLA tests free of charge and for 
practice only should have had considerable appeal. In Institutions In the State of Penn- 
sylvania, the results of the testing done In this project were, however, used as a part of 
regular certification procedures, by ari^angement with state authorities; It is therefore 
likely that a greater proportion of teache'r preparation candidates took the tests In that 
state than In other states, since they were apprised of this arrangement. 

] 

After the elimination of "odd" cases (see Chapter III) , there remained approximately 
1000 "regular" Cc^ses who had taken one or more of the Teacher Preparation tests of the MLA 
battery. Norms based on these cases have already been given In table 4.3. A recapitula- 
tion of the norm data Is given In Table 10.1, together with comparative data drawn from 
various publications of Educational Testing Service concerning the performance of NDEA In- 
stitute Teachers on these tests. Typically, the means of the CFLT samples are midway be- 
tween NDEA pretest and posttest means on each of the three tests . The foreign language 
majors were better prepared, on the whole, than teachers arriving at NDEA Institutes. It 
Is perhaps too much to expect the CFLT samples to be able to surpass the performance of 
the NDEA teachers after the latter had had the special training afforded them In the NDEA 
Language Institutes . 

For most of the analyses presented In this chapter, the samples used Included both 
"regular” and "odd" cases in French, German, Russian, and Spanish whose data were complete 
on all seven tests, i.e., both the teacher preparation tests and the skills tests. There 
were 985 such students. 

As contrasted with students who took none of the Teacher Preparation tests, students 
who did take the tests tended to be somewhat Inferior In MLA skills test performance. The 
comparisons are shown in Table 10.2, where it may be seen that mean MLA skills test scores 
of French and Spanish teacher preparation cases were significantly below those of French 
and Spanish students who did not take these additional tests. The comparisons of means 
were not significant, however, for the German and Russian groups. The results for French 
and Spanish groups agree with findings in Chapter VI where it was shown that students in- 
tending to teach in elementary or secondary school tended to have somewhat lower MLA 
skills test scores than those intending to teach at the college level, or not intending to 
teach . 

There was also a tendency for the CFLT samples to be more homogeneous than the NDEAV 
samples on the Teacher Preparation tests, that is, the standard deviations of the scores 
were in nearly every case smaller. (The exceptions were the comparisons with the standard 
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Table 10.1 

Summary Statistics for MLA Teacher Preparation Tests, 
Norm Groups (From Table 4.3 ), compared with 
Data from NDEA Institute Standardization Groups^ 
(ETS, 1961a, 1961b, 1962a, 1962b, 1962c, 1962d, 1964a) 



Language Group 


MLA Teacher Preparation Tests 

Applied Linguistics Cult. & Civ. Prof. 

(MIA-5) (MIA-6) 

N X a N X a N 


Preparation 

(MLA-7) 

X a 


FRENCH 


CFLT 




445 . 


.48.66 


7.23 


460 


46.10 


6.46 


4352 


62.77 


7.11 




NDEA 


Pretest 


C.700 


45.46 


8.40 


c.700 


44.33 


8.95 


1336 


59.1 


1,4 




NDEA 


Posttest 


c*700 


50.70 


8.42 


c.700 


50.15 


8.94 


1336 


66.9 


6.7 


GERMAN 


CFLT 




116 


49.88 


7.28 


111 


50 c 93 


8.75 


128^ 


62.52 


7.19 




NDEA 


Pretest 


C.150 


48.04 


9.68 


c.150 


48.62 


10.11 


297 


60.2 


7.6 




NDEA 


Posttest 


C.150 


54.64 


9.30 


c.150 


53.97 


9.06 


297 


67.3 


6.2 


ITALIAN 


CFLT 




8 


49.38 


4.95 


8 


46.12 


6.45 


(Not 


; computed) 




NDEA 


Pretest 


C.65 


46.1 


6.5 


c.65 


49.8 


8.0 


40 


53.9 


6.2 




NDEA 


Posttest 


C.65 


50.7 


7.8 


c.65 


54.5 


8,5 


40 


61.9 


6.7 


RUSSIAN 


CFLT 




18 


49.11 


6.25 


20 


52.80 


8.17 


23^ 


60.04 


5.26 




NDEA 


Pretest 


c.llO 


44.25 


6.79 


C.llO 


49.39 


9.15 


176 


59.7 


8.5 




NDEA 


Posttest 


c.llO 


48.45 


7.84 


C.llO 


53.42 


8.86 


176 


64.4 


7.0 


SPANISH 


CFLT 




394 


46.68 


7.43 


405 


55.54 


7.66 


3992 


61.17 


7.83 




NDEA 


Pretest 


c.700 


43.07 


7.69 


c.700 


49.53 


9.82 


1334 


58.8 


8.1 




NDEA 


Posttest 


c.700 


48.42 


8.51 


c.700 


55.95 


9.42 


1334 


66.8 


6.4 


ALL 


CFLT 
















991 


62.13 


7.43 




NDEA 


Pretest 














1745 


59.32 


8.10 




NDEA 


Posttest 














1709 


66.25 


6.81 


Totals 


CFLT 




981 






1004 






991 







^NDEA data on Form A (then labelled JML-1) are given, except for MLA-7 in the specific 
language groups, where the data are from both Form A and Form B combined because of the 
spiraling of these forms through institutes and through pretest and posttest administra- 
tions. 

^These cases were those used in certain other tabulations and may include some cases 
that for some reason were not used in the norm groups. They were complete on all seven 
MLA tests. 
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Table 10.2 



Comparisons on HLA Skills Tests of Those Who Did and Did Not 
Take Any Teacher Preparation Tests 



MLA Skills Test Scores 



Listening 



Speaking 



Readiaig 



Writing 







N 


% 


X 


a 


X 


a 


X 


a 


X 


a 












FRENCH MAJORS 










Took MLA 5-7 




A35 


32.2 


A1.98 


7.68 


72.80 


9.23 


A6.32 


8. A3 


A5.5A 


9.63 


Took none of 


these 


735 


62.8 


A5.68 


7.2A 


.A. 59 


9.6A 


50.58 


8. 6 


A8.50 


9.50 


Total 


1170 


100.0 


AA.31 


7.63 


73.92 


9.5A 


A9.00 


8.6A 


A7.39 


9. 66 


t 








-8. 


2** 


-3 


.1** 


-8. 


4** 


-5. 














C 


;?,RMAN MAJORS 










Took MLA 5-7 




128 


33.0 


AA.26 


8.90 


90.03 


15.62 


52.08 ' 


io.A7 


5A.A2 


13.51 


Took none of 


these 


260 


67.0 


A5.07 


8.55 


89.62 


13.77 


52.97 


9.88 


5 A. 26 


13.57 


Total 




388 


100.0 


AA.80 


8.69 


89.76 


1A.A2 


52.68 


10.10 


5A.31 


13.57 


t 








-0. 


,9 


0 


.3 


-0, 


,8 


0. 


,1 












RUSSIAN MAJORS 










Took MLA 5-7 




23 


22.5 


A3. 57 


7.16 


85. 7A 


16.17 


AA.87 


12,33 


62.57 


11.89 


Took none of 


these 


79 


77.5 


A3. 20 


5.91 


79. A2 


lA.OO 


A2.90 


10.75 


63.63 


10.99 


Total 




102 


100.0 


A3. 28 


6.2A 


80. 8A 


1A.S3 


A3.3A 


11.21 


63.39 


11.26 


t 








0, 


.2 


1 


.8 


0, 


.7 


-0 


.A 












SPANISH MAJORS 










Took MLA 5-7 




399 


AA.3 


A3. 75 


6.79 


8A.09 


12. A3 


A6.96 


8.52 


5A.2A 


10. A6 


Took tione of 


these 


501 


55.7 


A5.9A 


6.77 


86.05 


12.99 


A9.91 


8.7A 


56. A3 


10. A5 


To’.;al 




900 


100.0 


AA.97 


6.87 


85.18 


12.79 


A8.60 


8.77 


55. A6 


10.52 


t 








-A 


.8** 


-2 


:.3* 


-5 


.1** 


-3 


.1** 



** p<.01, * p<.05 
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deviatlons o£ the I)DEA posttest data on the Professional Preparation test; It is possible 
however, that these smaller standard deviations In the NDEA data were due to the nearer 
approach o£ the mean to the celling o£ the test, or to a greater homogeneity o£ the 
training experiences o£ the NDEA Institute teachers after the Institute courses.) This 
finding parallels that for the MLA skills test scores, discussed earlier (see Chapter IV). 



2. Estimated National Norms for Teacher Preparation Tests 

By procedures similar to those used In Chapter V for the MLA Skills tests, esti- 
mated national norms for the MLA Teacher Preparation Tests were developed. They are 
shown In Tables 10.3 and 10.4. The estimates were made on the {assumption, which Is mani- 
festly false, that all foreign language majors were enrolled In teacher preparation pro- 
grams and would be candidates for taking the Teacher Preparation tests. Nevertheless, 
the estimated norms are the best estimates available for the national population of 
teacher preparation candidates. Actually, the disparity between the samples used for 
normlng the MLA skills tests and those used for the Teacher Preparation tests affects 
the Interpretation of the skills test scores for teacher preparation candidates. A proper 
set of norms foi all seven MLA tests applicable to teacher preparation candidates would 
be based solely on samples u£ those candidates. Use of the estimated norms for MLA skills 
tests In Chapter V will tend to give a slight handicap to teacher preparation candidates 
because at least In the case of French and Spanish majors they are somewhat Inferior, 
on the average, to students not enrolled In teacher preparation programs, as shown In 
Table 10.2. On the other hand. It Is perhaps useful to handicap teacher preparation can- 
didates In this way to emphasize their tendency to be Inferior In basic language skills. 

3. In ter correlations of MLA Teacher Preparation 
Tests and Correlations with MLA Skills Tests 



Intercorrelatlons of the Teacher Preparation tests and correlations with Skills tests^ 
for the groups of French, German, Russian, and Spanish majors who took the former tests 
and also had complete data on the Skills tests are shown In Table 10.5, with comparative 
data from various documents published by Educational Testing Service (ETS, 1961a, 1961b, 
1962a, 1962b) concerning these correlations In NDEA Language Institute samples. The 
Intercorrelatlons between the Applied Linguistics test and the Civilization and Culture 
test range from .43 to .58, reflecting possibly the general level of training achieved by 
the student. The correlations of these two tests with the Professional Preparation test 
are at about the same level for the Applied Linguistics test (correlations of .50, .51, 

.19, and .57, the correlation of .19 being for only 23 cases In Russian), and slightly 
lower for the Culture and Civilization test (correlations of .37, .30, -.12, and .34). 

One may be fairly well assured that the three Teacher Preparation Tests measure largely 
different aspects of professional teaching competence. 

From the data shown, the Applied Linguistics and the Culture and Civilization tests 
show moderate correlations with MLA Skills tests, again reflecting, no doubt, the general 
level of training In the foreign language and associated wisdom achieved by the student. 
Correlations of the Professional Preparation test with MLA Skills tests tend to be much 
lower; this test, after all, has little or nothing to do with the knowledge of specific 
languages since It Is all In English and concerns only the principles and procedures 
whereby any foreign language may be taught. 

If account Is taken of the differences In the sizes of the samples and the homogene- 
ity of the groups, the results for the CELT samples are not Inconsistent with the data 
from the NDEA Language Institute teachers. 
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Table 10.3 . 



Estimated National Norms for MLA Teacher Preparation Tests, 
College Senior FL Majors (Percentile Ranks of Converted Scores) 



FRENCH 



Applied 

Linguistics 


Culture and 
Civilization 




%ile 




%lle 


Score 


Rank 


Score 


Rank 


69 


99.9 


67 


99.8 


68 


99.7 


66 


99.5 


67 


99.6 


65 


99.5 


66 


99.1 


64 


99.4 


65 


98.7 


63 


99.2 


64 


98.5 


62 


99.1 


63 


97.8 


61 


98.5 


62 


96.9 


60 


97.9 


61 


95.8 


59 


97.2 


60 


94.4 


58 


95.8 


59 


92.6 


57 


94.2 


58 


90.7 


56 


92.1 


57 


88.9 


55 


90.2 


56 


85.5 


54 


88.4 


55 


80.5 


53 


85.7 


54 


77.1 


52 


82.3 


53 


74.1 


51 


79.3 


52 


70.4 


50 


77.0 


51 


66.4 


49 


72.7 


50 


61.5 


48 


66.1 


49 


56.7 


47 


58.4 


48 


52.3 


46 


50.3 


47 


46.8 


45 


44.3 


46 


39.5 


44 


39.2 


45 


33.4 


43 


33.4 


44 


28.9 


42 


26.8 


43 


24.3 


41 


20.3 


42 


20.8 


40 


15.7 


41 


17.3 


39 


12.3 


40 


13.5 


38 


9.8 


39 


10.0 


37 


8.4 


38 


6.9 


36 


7.0 


37 


4.8 


35 


4.8 


36 


3.6 


34 


3.1 


35 


2.6 


33 


2.3 


34 


1.7 


32 


1.3 


33 


1.1 


31 


0.8 


32 


0.4 


30 


0.6 






29 


0.4 






28 


0.2 



GERMAN 

Applied Culture and 



Linguistics 


Civilization 




%ile 




%lle 


Score 


Rank 


Score 


Rank 


69 


99.6 


70 


99.8 


68 


99.3 


69 


99.1 


67 


99.3 


68 


98.6 


66 


99.3 


67 


98.4 


65 


99.0 


66 


97.0 


64 


97.7 


65 


94.4 


63 


96.6 


64 


92.1 


62 


96.1 


63 


90.5 


61 


95.5 


62 


89.2 


60 


95.0 


61 


87.6 


59 


93.9 


60 


86.2 


58 


91.5 


59 


83.6 


57 


88.2 


58 


79.0 


56 


84.0 


57 


75.4 


55 


79.2 


56 


73.2 


54 


76.5 


55 


69.9 


53 


72.6 


54 


66.6 


52 


68.7 


53 


65.0 


51 


63.5 


52 


62.7 


50 


57.0 


51 


58.6 


49 


54.8 


50 


54.8 


48 


52.9 


49 


52.3 


47 


50.2 


48 


47.8 


46 


45.5 


47 


38.9 


45 


38.1 


46 


31.5 


44 


27.7 


45 


29.0 


43 


18.5 


44 


24.1 


42 


14.9 


43 


19.4 


41 


12.5 


42 


16.2 


40 


8.7 


41 


12.9 


39 


5.5 


40 


11.0 


38 


3.7 


39 


9.8 


37 


1.7 


38 


8.8 


36 


0.4 


37 


6.0 






36 


4.2 






35 


2.1 



RUSSIAN (tentative) 



Applied Culture and 
Linguistics Civilization 





%ile 




%lle 


Score 


Rank 


Score 


Rank 


64 


97.9 


71 


97.2 


63 


95.8 


70 


94.3 


62 


95.8 


69 


94.3 


61 


95.8 


68 


94.3 


60 


95.8 


67 


94.3 


59 


95.8 


66 


94.3 


58 


92.4 


65 


94.3 


57 


88.9 


64 


92.2 


56 


86.8 


63 


88.0 


55 


81.3 


62 


85.9 


54 


77.8 


61 


83.7 


53 


77.8 


60 


81.6 


52 


77.8 


59 


81.6 


51 


72.3 


58 


81.6 


50 


59.0 


57 


78.6 


49 


47.7 


56 


72.6 


48 


43.1 


55 


66.7 


47 


41.8 


54 


63.6 


46 


36.3 


53 


61.5 


45 


30.0 


52 


53.2 


44 


19.0 


51 


44.0 


43 


7.8 


50 


34.7 


42 


6.9 


49 


28.5 


41 


6.9 


48 


28.5 


40 


3.4 


47 


21.4 


. 




46 


14.2 






35 


14.2 






44 


11.1 






43 


8.0 






42 


5.0 






41 


1.9 






40 


1.9 






39 


1.9 






38 


0.9 
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Table 10.3 (continued) 



SPANISH 



Applied 



Culture and 



Professional Preparation 
(All Languages) 



Llnguxstlcs 


Civilization 








%ile 




%ile 




%ile 


Score 


Rank 


Score 


Rank 


Score 


Rank 


70 


99.9 


73 


99.9 


80 


99.9 


69 


99.8 


72 


99.0 


79 


99.9 


68 


99.8 


71 


98.1 


78 


99.7 


67 


99.8 


70 


97.5 


77 


99.4 


66 


99.4 


69 


96.4 


76 


99.0 


65 


99.0 


68 


95.4 


75 


98.1 


64 


98.9 


67 


94.5 


74 


96.9 


63 


98.6 


66 


91.5 


73 


95.3 


62 


97.9 


65 


87.8 


72 


92.9 


61 


96.9 


64 


84.6 


71 


89.8 


60 


95.8 


63 


80.2 


70 


86.1 


59 


94.4 


62 


76.5 


69 


82.0 


58 


92.7 


61 


73.3 


68 


78.2 


57 


90.9 


60 


69cl 


67 


74.1 


56 


89.6 


59 


64.9 


66 


68.9 


55 


88.5 


58 


61.2 


65 


62.8 


54 


86.3 


57 


57.5 


64 


57.9 


53 


83.2 


56 


52.7 


63 


53.4 


52 


79.6 


55 


47.1 


62 


48.1 


51 


75.8 


54 


42.2 


61 


42.6 


50 


72.2 


53 


38.6 


60 


37.2 


49 


67.3 


52 


35.5 


59 


32.7 


48 


62.3 


51 


30.5 


58 


28.9 


47 


57.4 


50 


25.6 


57 


24.9 


46 


52.6 


49 


22.2 


56 


20.8 


45 


48.1 


48 


18.9 


55 


16.7 


44 


41.5 


47 


15.7 


54 


13.4 


43 


34.1 


46 


13.0 


53 


11.1 


42 


28.7 


45 


10.9 


52 


9.1 


41 


24.5 


44 


9.1 


51 


7.2 


40 


19.2 


43 


6.9 


50 


5.8 


39 


15.4 


42 


4.7 


49 


4.8 


38 


13v3 


41 


3.5 


48 


4.1 


37 


10)^7 


40 


2.4 


47 


3.4 


36 


7.4 


39 


1.1 


46 


2.7 


35 


4.6 


38 


0.2 


45 


2.2 


34 


3.4 


37 


0.2 


44 


1.9 


33 


1.9 


36 


0.1 


43 


1.6 


32 


0.4 






42 


1.3 










41 


1.1 










40 


0.7 










39 


0.5 










38 


0.5, 










37 


0.4 










36 


0.3 










35 


0.3 










34 


0.2 










33 


0.2 










32 


0.2 










31 


0.2 










30 


0.1 










29 


0.0 



l'1 

u > 



187 



Table 10 « 4 

Estimated College Senior Major National Norms 
MLA Teacher Preparation Tests 
Converted Scores, for Selected Percentile Ranks 



FRENCH(Est.N-5564)GERMAN(E3t.N-1848)RUSSIAN>fEst.N-365)SPANI^H(Est.N-4439) 
Percentile Appl. Cult,& Appl, Cult.& Appl« Cult,& Appl, Cult«& 



Rank 


Ling. 


Civ. 


Ling. 


Civ. 


Ling. 


Civ. 


Ling. 


Civ. 


99 


66 


62 


65 


69 


64 


71 


65 


■ 


97 


62 


59 


63 


66 


64 


71 


61 


70 


95 


60 


38 


60 


65 


61, 


70 


59 


68 


90 


58 


55 


58 


63 


57 


63 


56 


66 


85 


56 


53 


56 


60 


56 


62 


54 


64 


80 


55 


51 


55 


58 


55 


57 


52 


63 


75 


53 


50 


54 


57 


51 


56 


51 


62 


70 


52 


49 


52 


55 


51 


56 


50 


60 


65 


51 


48 


51 


53 


51 


55 


49 


59 


60 


50 


47 


51 


51 


50 


53 


48 


58 


55 


49 


47 


49 


50 


50 


52 


46 


56 


50 


48 


46 


47 


48 


49 


52 


45 


56 


45 


47 


45 


46 


48 


48 


51 


45 


55 


40 


46 


44 


45 


47 


47 


51 


44 


53 


35 


45 


43 


45 


46 


46 


50 


43 


52 


30 


44 


42 


44 


45 


45 


49 


42 


51 


25 


43 


42 


44 


44 


45 


48 


41 


50 


20 


42 


41 


43 


43 


44 


47 


40 


48 


15 


41 


40 


42 


42 


44 


46 


39 


47 


10 


39 


38 


40 


39 


43 


44 


37 


44 


5 


37 


35 


39 


36 


41 


42 


35 


42 


3 


35 


34 


38 


35 


40 


41 


34 


41 


1 


33 


31 


37 


35 


40 


38 


33 


39 


No. cases actually tested 


445 


460 


116 


111 


18 


20 


393 


404 


Mean (est. norms) 


48.18 


45.97 


48.62 


50.15 


49.16 


52.80 


46.18 


55.42 


S.D. (est. nonns) 


[7.161 




[6.781 


rs.431 


.. [5.7.3] 


— U*MJ_ 


[7»191 


[7.861 



Percentile Professional Preparation .(Est. N>12,362) 

Rank (All languaees) 



99 


76 


97 


74 


95 


73 


90 


71 


85 


70 


75 


67 


70 


66 


65 


65 


60 


64 


55 


63 


50 


62 


h 5 


61 


40 


60 


35 


59 


30 


58 


25 


57 


20 


56 


15 


55 


10 


52 


5 


49 


3 


46 


1 


41 


No. cases actually tested 


982 


Mean (est. norms) 


61.91 


S.D. (est. norms) 


[7.44J 


^Tentative because 


of small N 
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Table 10.5 

Intercorrelatlons of. MLA Teacher Preparation Tests and Correlations with 
MLA Skills Tests, for CFLT Samples as Compared with Results from 
NDEA Institute Standardization Samples (ETS, 1961a, 1961b, 1962a, 1962b) 



FRENCH GROUPS TAKING THE TEACHER PREPARATION TESTS 
CFLT N » 435 NDEA N = 1336 



MLA Skills Tests 







5 


6 


7 


L 


S 


R 


W 


Applied 

Linguistics 


5 


1.00 

[.782] 


.50 

(.593) 


.50 

(.542) 


.52 

(.581) 


.54 

(.523) 


.57 

(.629) 


.62 

(.655) 


Civilization 
and Culture 


6 


.50 

(.593) 


1.00 

[.741] 


.37 

(.491) 


.53 

(.528) 


.46 

(.460) 


.63 

(.624) 


.55 

(.557) 


Professional 

Preparation 


7 


.50 

(.542) 


.37 

(.491) 


1.00 

[.668] 


.39 

(.306) 


.30 

(.198) 


.42 

(.323) 


.39 

(.296) 




Mean 


48.87 


46.42 


62.77 


41.98 


72.80 


46.32 


45.54 




(50.8) (49.8) < 


[66.9) 


(42.3) 


(84.1) 


(46.7) 1 


(46.5) 




S.D. 


7.09 


6.37 


7.11 


7.69 


9.25 


8.44 


9.64 






(8.6) 


(8.9) 


(6.7) 


(8.7) 


(18.6) 


(10.5) 1 


(12.5) 






GERMAN GROUPS TAKING THE 


TEACHER PREPARATION TESTS 








CFLT N = 


128 






NDEA 


N = 297 














MLA Skills Tests 








5 


6 


7 


L 


S 


R 


W 


Applied 

Linguistics 


5 


1.00 

[.854] 


.49 

(.671) 


.51 

(.437) 


.35 

(.533) 


.36 

(.561) 


.39 

(.616) 


.49 

(.750) 


Civilization 


6 


.49 


l.CO 


.30 


.49 


.48 


.54 


•55 


and Culture 




(.671) 


[.■30] 


(.405) 


(.502) 


(.454) 


(.583) 


( .605) 


Professional 


7 


.51 


.30 


1.00 


.17 


.09 


.20 


.19 


Preparation 




(.437) 


(.405) 


[.635] 


(.191) 


(.141) 


( . 178) 


( . 248) 




Mean 


49.60 


51.34 


62.52 


44.26 


90.03 


52.08 


54.42 




(53.8) 


(54.0) 


(67.3) 


(42.2) 


(84.5) 


(48.7) 


(48.9) 




S.D. 


7.20 


8.58 


7.19 


8.94 


15.68 


10.51 


13.56 






(9.2) 


(9.2) 


(6.2) 


(8.6) 


(17.7) 


(10.5) 


(15.1) 



RUSSIAN GROUPS TAKING THE TEACHER PREPARATION TESTS 
CFLT N = 23 NDEA N = 176 



MLA Skills Tests 







5 


6 


7 


L 


S 


R 


W 


Applied 

Linguistics 


5 


1.00 

[.771] 


.58 

(.535) 


.19 

(.444) 


.62 

(.432) 


.53 

(.362) 


.56 

(.394) 


.79 

(.676) 


Civilization 


6 


.58 


1.00 


-.12 


.66 


.48 


.62 


.70 


ana Cultvire 




(.535) 


[.750] 


(.372) 


(.429) 


(.271) 


(.331) 


(.504) 


Professional 


7 


.19 


-.12 


1.00 


-.17 


-.07 


-.19 


-.01 


Preparation 




(.444) 


(.372) 


[.686] 


(.219) 


(.114) 


(.033) 


(.210) 




Mean 


50.22 


53.57 


60.04 


43.57 


85.74 


44.87 


62.57 




(48.8) 


(53.1) 


(64.4) 


(41.7) 


(77.5) 


(38.2) 


(54.9) 




S.D. 


6.45 


8.06 


5.26 


7.32 


16.54 


12.60 


12.15 






(6.7) 


(8.5) 


(7.0) 


(7.3) 


(18.0) 


(11.1) 


(14.3) 



Table 10.5, continued 



SPANISH GROUPS TAKING THE TEACHER PREPARATION TESTS 



CELT N - 399 







5 


6 


Applied 

Linguistics 


5 


1.00 

[.775] 


.43 

(.570) 


Civilization 
and Culture 


6 


.43 

(.576) 


1.00 

[.780] 


Professional 

Preparation 


7 


.57 

(.627) 


.34 

(.548) 




Mean 


46.52 

(49.0) 


55.53 

(56.4) 




S.D. 


7.36 

(8.5) 


7.74 

(9.0) 



NDEA, N * 1334 



MLA Skills Tests 



7 


L 


S 


R 


W 


.57 

(.627) 


.32 

(.502) 


.09 

(.410) 


.31 

(.559) 


.38 

(.623) 


.34 

(.548) 


.46 

(.606) 


.28 

(.517) 


c52 

(.673) 


.52 

(.681) 


1.00 

[.701] 


.22 

(.383) 


.05 

(.271) 


.16 

(.419) 


.23 

(.457) 


61.17 

(66.8) 


43.75 

(41.6) 


84.09 

(77.5) 


46.96 

(44.1) 


54.24 

(48.7) 


7.83 

(6.4) 


6.80 

(7.8) 


12.44 

(18.9) 


8.53 

(9.5) 


10.47 

(12.3) 



^NDEA data, given in parentheses, are from posttests, except that self-correlations, 
given in brackets [ ] , are correlations between pre-test and post-test of different forms 

as an estimate of reliability. 
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Variables associated with performance on thie MLA Teacher Preparation Tests 

Various tabulations were made in an effort to investigate what variables might be 
associated with performance on the Teacher Preparation tests. Since foreign language 
aptitude had been shown to be relevant in explaining some of the variance in the MLA 
skills test scores, this was investigated also in connection with the Teacher Preparation 
tests, with results shown in Table 10.5. A considerable number of significant correla- 
tions appear in that table, the Incidence of significant correlations being largely de- 
pendent upon the numbers of cases involved. In French and Spanish, the Applied Linguist- 
ics test is best predicted by the MLAT Total Score, but the MLAT Total shows significant 
correlations also with Culture and Civilization and with Professional Preparation test 
scores. In German, with a smaller number of cases, the highest correlation with MLAT 
Total is for the Professional Preparation test. When we examine the correlations for MLAT 
subtests, it becomes, clear that there is a specific relation, in all languages, between 
MLAT-A (Words in Sentences, a test of "grammatical sensitivity") and the Applied Linguist- 
ics test; the correlations range from .25 (non— significant, for Russian) to .49 (highly 
significant, for Spanish) . These results seem reasonable, although there was no particu- 
lar reason for expecting them; students with special language aptitude, especially in 
grammatical sensitivity, are more likely to develop Interests in the linguistic facts and 
principles tested in the Applied Linguistics test, and more generally in the knowledges 
tested in the Culture and Civilization and the Professional Preparation tests. In addi- 
tion it must be pointed out that although the MLAT was designed as a special language 
aptitude test, its scores are to some extent a reflection of general intelligence, and 
this may be the reason that MLAT scores show some correlation with scores on the Profess- 
ional Preparation test, which probably is correlated with general scholastic achievement. 

The educational level at which the student Intended to teach was also a variable 
associated with performance on the Teacher Preparation tests, as may be seen in Table 10.7. 
Bie major trend in this table Is the one that we have come to expect from previous find- 
ings, namely, that those who include teaching at the college level in their plans do 
better on the MLA tests than those who aspire only to teaching in elementary or secondary 
school. This trend was significant for all three Teacher Preparation tests for the French 
group, but only for the Civilization and Cu ture test in the case of the Spanish group. 

(Insufficient numbers of cases were available in German and Russian to justify this type 
of analysis.) 



It will be noted in Table 10.7 that the large majority of those who took the teacher 
prepay ation tests were students intending to teach at the high school level. This is due 
partly to the restriction placed upon taking the Teacher Preparation tests; only students 
enrolled in programs "expressly oriented toward the preparation of elementary or secondary 
school language teachers" were supposed to take the tests. It can be inferred, by com- 
paring the figures in Table 10.7 with those in Table 6.7 that many individuals intending 
to teach only at the college level, and not enrolled in teacher preparation programs, were 
not allowed to take the Teacher Preparation tests. Therefore, the norms developed here 
for the Teacher Preparation tests would have to be used with considerable caution if they 
are applied to individuals intending teaching at the college level. 

lime of beginning foreign language study had been shown to be an Important variable 
in explaining variation in performance on MLA Skills tests. The students who took the 
Teacher Preparation tests were classified according to whether they had begun the studr of 
their major foreign language in grade school, high school, or college, and the means and 
standard deviations of the scores are shown in Table 10.8. Although there were slight 
trends similar to those found for the Skills tests, they were weak and only rarely of stat- 
istical significance st even the 5% level. (The table gives the exact levels of signifi- 
cance for each comparison.) The results suggest that starting language study early gives 
only a slight advantage to the student in acquiring those aspects of teacher competence 
that are tested by the Teacher Preparation tests. It is likely that many of these aspects 
arc learned mainly at the college level, in any event. For example, it is unlikely that 
the student starting foreign language study at the elementary or secondary school level 
would be Introduced to, or made aware of, many of the specific facts and principles of 
applied linguistics until he reaches the college level. Likewise, it would appear that 
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Table 10.6 

Correlations of MLAT Subtests with MLA Teacher Preparation Tests 



Total 



FRENCH (N = 408) 



MLAT Subtest 


Mean 


S.D. 


3 (Spelling Clues) 


26.80 


9.23 


4 (Words in Sentences) 


32.48 


5.90 


5 (Paired Associates) 


.19.99 


4.62 



79.26 



13.91 



Correlations 



MLA-5 



.31** 

.39** 

.27** 



.46** 



MLA-6 



.27** 

.25** 

.25** 



.37** 



MLA-7 



.16** 

.25** 

.18** 



.27** 



MLAT Subtest 

3 (Spelling Clues) 

4 (Words in Sentences) 

5 (Paired Associates) 



Total 



79.12 



16.91 



Mean 


GERMAN (N - 102) 

S.D. 


MLA-5 


Correlations 

MLA-6 


MLA-7 


25.56 


10.12 


.22* 


.12 


.27** 


33.32 


6.12 


.34** 


.04 


.29** 


20.24 


5.06 


.13 


-.04 


.29** 



.29** 



,07 



.35** 



RUSSIAN (N - 17) 



3 (Spelling Clues) 

4 (Words in Sentences) 



3 (Spelling Clues) 

4 (Words in Sentstnces) 

5 (Paired Associates) 



Correlations 




26.94 

30.41 



12.12 

7.94 

3.63 



-.10 

.25 

.14 



.21 

.28 

-.08 



-.32 

-.02 

.26 



Total 




78.59 


19.88 .06 


.23 -.16 




O 




SPANISH (N - 352) 


Correlations 


MLAT Sub test 




Mean 


S.D. MLA-5 


MLA-6 MLA-7 



Total 



24.95 


10.36 


.19** 


.05 


.10 


30.64 


7.58 


.49** 


.28** 


.39** 


18.98 


5.61 


.22** 


.17** 


,20** 


74.31 


16.65 


.51** 


.33** 


.43** 



**p < .01; *p < .05 for hypothesis of zero correlation 
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Table 10.7 

Means and Standard Deviations of MLA Teacher Preparation Test 
Scores, by Intentions to Teach at Various Levels, for 
French and Spanish Groups 





M 


% 


Applied 
Linguistics 
_MLA-5 
X a 


Culture & 
Civilization 
MLA-6 
X a 


Professional 
Preparation 
MLA- 7 
X a 






FRENCH MAJORS 




- 




Do not intend to teach 


18 


4.2 


49.94 8.82 


47.33 8.55 


56.78 


9.36 


Intend to teach at: 














Elem. sch. only 


13 


3.1 


45.54 7.76 


41.31 6.11 


61.23 


6.65 


Elem. sch. or high sch. 


29 


6.8 


46.72 6.63 


47.10 6.42 


62.31 


5.66 


High sch. only 


302 


71.2 


48.31 6.92 


45.55 6.02 


62.74 


6.97 


High sch. or college 


50 


11.8 


52.04 7.44 


49.20 6.56 


64.38 


6.88 


College only 


12 


2.8 


48.42 6.92 


45.83 6.68 


61.92 


10.20 


Total 


424 


99.9 


48.63 7.23 


46.04 6.44 


62.58 


7.24 


F 






3.44 


4.65 


3. 


16 


P 






< .01 


< .001 


< . 


01 






SPANISH MAJORS 








Do not Intend to teach 


15 


4.1 


43.33 5.86 


53.67 7.26 


57.73 


7.57 


Intend to teach at: 














Elem. sch. only 


— 












Elem. sch. or high sch. 


20 


5.4 


47.00 7.26 


53.60 7.49 


62.80 


7.36 


High sch. only 


275 


74.9 


46.37 7.09 


54.93 7.36 


61.37 


7.59 


High sch. or college 


40 


10.9 


47.50 8.35 


57.75 8.13 


. 60.80 


8.15 


College only 


17 


4.6 


47.12 6.80 


60.35 8.78 


59.06 


10.38 


Total 


367 


99.9, 


46.44 7.23 


55.37 7.67 


61.13 


7.85 


F 






0.98 


3.52 


1. 


31 


P 






> .05 


< .01 


> . 


05 
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Table 10.8 

i 

Mean MLA Teacher Preparation Test Scores, by Language and 
By Time of Beginning Study of the Major Language 







N 


% 


Teacher Preparation Tests 


Appl. Ling. 


Cult. & Civ. 


Prof. Prep. 


X or 


X a 


X a 










FRENCH 






Began 


In grade school 


41 


9.5 


49.07 5.18 


45.68 6.44 


60.29 6.73 


Began 


In high school 


313 


72.3 


49.01 7.43 


46.44 6.41 


63.03 7.23 


Began 


In college 


79 


18.2 


47.20 7.02 


44.96 6.39 


62.35 7.11 






433 


100.0 


48.68 7.21 


46.10 6.45 


62.65 7.22 




^2, 430 






2.05 


1.76 


2.71 




P 






.13 


.17 


.07 










GERMAN 






Began 


In high school 


58 


55.2 


50.76 7.16 


52.05 8.40 


64.31 7.21 


Began 


In college 


47 


44.8 


48.34 7.07 


49.45 8.57 


61.96 6.14 






105 


100.0 


49.68 7.25 


50.89 8.62 


63.26 6.88 




^1, 103 






2.94 


2.40 


3.09 




P 






.08 


.12 


.08 










SPANISH 






Began 


In grade school 


39 


10.1 


49.03 7.63 


57.44 6.54 


62.15 9.50 


Began 


In high school 


272 


70.1 


46.68 7.47 


55.23 7.72 


61.56 7.72 


Began 


In college 


77 


19.8 


45.32 6.55 


55.14 7.91 


60.08 7.47 






388 


100.0 


46.65 7.38 


55.43 7.69 


61.32 7.90 




^2, 385 






3.30 


1.48 


1.29 




P 






.04 


.23 


.28 



V 




KOlHkiMiH 
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students acquire their erudition in Culture and Civilisation mainly at the college level. 
(This conclSsimi is supported by the findings to he discussed next.) 

TO investigate other verifies recourse was f 

j T*ACTT*6'Ssi.on introduced in CVmpter VIII# The innj a^a -iti 

canonical regression mt a cir-fiics test scores were included in 

^Se^: -^;“l1u^i?^fnTahr^^^^ ‘’^hf^a^ ^sl:^^%rL:its Of both the multi- 

pie regresLon analyses and the canonical regression analyses. 

T 4 . » first the multiple regression analyses, in the upper part of the 

«- M Each of the MIA Teacher Preparation tests is predictable to a highly significant 
efte«; fr™ info^^t?^ ^ st^dent background, training history ^d 
r^/?:?al score^is generally 

cofrelafi™; would dou^JL^tave been significantly higher.) Amount of 

i^n aoes^ It is somewhat surprising, however, that number of hours in miscellaneous 
courses” shows only small and generally non-significant beta-weights. *^°'*i^^^flsc'*in 

Prepa«tion tesL cannot be ascribed to homogeneity of the teacher P«P“f 

The following data will support these observations: 



Hniir.ct of Miscellaneous Courses 



Teacher Preparation Mean 

Sample S.D. 

N 

Complete CFLT Mean 

Samples as used S.D. 

in Table 8.2 N 



FRENCH 


GERMAN 


SPANISH 


3.35 


3.58 


3.45 


4.31 


3.68 


4.57 


397 


98 


343 


3.06 


2.60 


3.44 


4.40 


3.57 


4.97 


1039 


389 


793 



There is perhaps some meaning in the fact that the German teacher preparation sample shms 
a consldeMbly'^hlgher mean number of hours of miscellaneous courses than ^he c^lete CFLT 
sample in Gerlan.^and also shows a significant beta-weight in predicting CiviUzation and 
S“?ure test scoies. Do German teacher preparation programs lay more stress on pedagog- 
ical courses than other programs? These results suggest so. 



The canonical regression results shown in Table 10.9 amplify and enrich the above 
interpretations. In German and Spanish, there are two independent 

that are significant at the 5% level or better, while in French one is highly significant 

significant at the 10% level. Nevertheless, ~ ^^-^^Thinr^at as 

the patterns of canonical regression weights across languages tempts one to think that as 
many as three canonicals are of interest. Sorting them, we re-label them A, B, and C. 

Canonical A is a summarization of the three Teacher Preparation tests with generally 
positive weights: in French, Applied Linguistics and Culture and Civilization are posi- 

tively weighted, while in German and Spanish, Applied Linguistics and Professional 
ation are emphasized. This combination of the criterion tests is J - 

dieted by the language aptitude test (MLAT Total), and seems to represent that portion of 
the Teacher Preparation test variance that is most closely associated with student traits 
constituting language aptitude and possibly interest. 
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Table 10.9 

Multiple and Canonical Regression Analyses for 
Teacher Preparation Tests and Selected Predictor Variables 



A. Multiple Regression Analysis (Beta Weights and Multiple .Correlations) 

Criterion Variables (MLA Tests) 



Predictor Variables 

Time Abroad 
Use of FL at Home 
Semesters Grade Sch. 
Semesters High Sch. 
Hrs. College Lang. 
Hrs . College Lit . 
Hrs. Misc. Courses 
MLAT Total 

Multiple R: 

N 



FRENCH MLA 



GERMAN MLA 



SPANISH MLA 



5 


6 


7 


5 


6 


7 


5 


6 


7 


.11* 


.20** 


.02 


.25* 


.29** 


.07 


.03 


.14* 


.02 


.09 


.07 


.06 


.08 


.14 


.07 


-.08 


.12* 


-.19** 


.03 


-.04 


-.10** 


,08 


-.06 


.09 


.07 


-.02 


.06 


.05 


.04 


.06 


.14 


.12 


.25* 


.10 


.03 


.02 


.04 


.00 


.10 


.23 


.12 


.26* 


.07 


.04 


-.01 


.03 


.13** 


.05 


.04 


.30** 


-.13 


-.04 


.27** 


.00 


.03 


.06 


-.01 


.08 


.19* 


-.03 


.06 


.07 


.01 


.45** 


.34** 


.25** 


.38** 


.18 


.38** 


.46* 


,29** 


.33** 


.49** 


.46** 


.30** 


.50** 


.57** 




.46** 


.51** 



.45** 


.42** 



397 



98 



343 



B. Canonical Regression Analysis (Canonical Regression Coefficients and R*s) 

Successive Canonical Regressions, by Language 



FRENCH 

Original No: I II HI 

Relabeled: ABC 



Predictors (Canonical Regression Weights) 



Time Abroad 


.33 


.64 


.01 


Uiie of FL at Home 


.18 


-.07 


.08 


Semesters Gra""J Sch. 


-.07 


.19 


-.65 


Semesters High Sch. 


.10 


-.14 


.18 


Hrs . College Lang . 


.06 


-.40 


.45 


Hrs. College Lit. 


.17 


.41 


.51 


Hrs. Misc. Courses 


.09 


.25 


— . 08 


MLAT Total 


.90 


-.37 


-.26 


Canonical R: 


.53 


.19 


.12 


PJ 


< .001 


.10 

1 


.41 


Criteria (Canonical Regression Weights) 




MLA-5 (Appl. Ling.) 


.78 


-.44 


-.66 


MLA-6 (Cult. & Civ.) 


.62 


.77 


.28 


MLA-7 (Prof. Prep.) 


.03 


-.46 


.70 





GERMAN 






SPANISH 




I 


II 


III 


I 


II 


III 


B 


A 


C 


A 


B 


C 



.57 


.07 


-.50 


.09 


.33 


.10 


.24 


.04 


.23 


-.19 


.52 


-.65 


-.11 


.24 


-.23 


.12 


-.16 


-.03 


.14 


.35 


.78 


.17 


-.05 


-.42 


.17 


.43 


.18 


.11 


.04 


-.39 


.58 


-.38 


-.02 


.01 


.76 


.39 


.37 


-.12 


-.09 


.12 


.12 


-.21 


.29 


.68 


-.05 


.94 


.08 


-.21 


.59 


.51 


.13 


.52 


.42 


.15 


< .001 


.01 


.95 


< .001 


< .001 


.23 



.29 


.50 


-.74 


.92 


-.33 


-.65 


.91 


-.47 


.33 


.17 


.91 


.08 


-.29 


.73 


.58 


.35 


-.25 


.76 



**p < .01; *p < .05. Significance levels given only for beta-weights, multiple 
correlations, and canonical correlations. 
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U4 u «« i-Vio rriterion side, features the Civilization and Cult- 
Canonical B is one which, on the three languages, it is predicted well 

ure test in contrast with the other tests. au-o-*! and Hours of College Literature. 

by two variables in var^les in’prench, the latter is the more im- 

The former is the more important of t weighted in German. This finding gives 

portant in Spanish, t two^ays^of acquiring suiierior knowledge of the cult- 

protracted attention to the literature of that language. 

Canonical C gives emphasis. ^ the “iterion side, to the^Pro^^^^ 

Test. It is a very weak source should be remembered that 

terns of canonical regression weights ' ession weights are scaled in 

despite the small canonical ® eoual to unity* it is therefore almost inevit* 

reliably predictable • 
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Chapter XI 



SUMMARY AND RECOMMENDATIONS 



1. Purpose and Design of the Study 

The purpose of this study was to assess the overall levels of foreign language pro- 
ficiency attained by senior class students ''majoring" or concentrating in foreign lang- 
uages in colleges and universities of the United States, and to identify factors strongly 
associated with these levels of competence, with the hope that such information might 
suggest ways in which foreign language teaching for these students could be Improved. 

The study was necessarily based upon a sample of graduating seniors majoring in for- 
eign languages drawn from the population of such majors and tested at a particular point 
of time. The target population chosen for this purpose was that of all students who were 
majoring in French, German, Italian, Russian, and Spanish and graduating in the class of 
1965 from colleges and universities in the United States. Mors precisely, the target 
population consisted of all students majoring in these languages and graduating from in- 
stitutions that had been listed in U. S. government publications as offering B. A. pro- 
grams in foreign langviages in the academic y^^'.ar 1962-63. In drawing the population, a 
two-stage stratified sampling design was employed in which institutions were to be 
sampled randomly within certain classifications as to size and principal source of fin- 
ancial support, and complete samples of students within these institutions were to be 
tested. The number of institutions to be sampled within size and institution type class- 
ifications WES determined by statistical formulas that specified a sample that promised 
to yield the maximum amount of information with the least amount of error for the planned 
budget . 

As measures of forelgvi language proficiency, as well as of certain aspects of pre- 
paration for teaching (for those students enrolled in programs for preparing teachers of 
foreign languages) , the subtests of Form A of the MLA Foreign Language Proficiency Tests 
for Teachers and Advanced Students were used: four tests of foreign language skills 

(Listening, Speaking, Reading, and Writing) and three tests of teacher preparation (Ap- 
plied Linguistics, Culture and Civilization, and Professional Preparation). As sources 
of subsidiary information on the students, th^ "short form" of the Modem Language Apti- 
tude Test and a Questionnaire for Foreign Language Majors were used. The latter inquired 
in considerable detail into the backgrounds and foreign language learning experiences of 
the students. A Questionnaire for Foreign Laneuage Department Chairmen sought to gain 
information on the characteristics of the college or university foreign language depart- 
ments in which the sample of students had studied. 

Parallel to the main study, an ancillary study was performed in an attempt to estab- 
lish meaningful equivalents for the skill levels measured by the MLA skills tests. For 
this purpose, the Foreign Service Institute interview procedure was administered to small 
samples of teachers attending NDEA Language Institutes, the resulting "absolute language 
proficiency ratings" being equated to scores attained on the MLA skills tests. These 
samples were chosen in such a way as to represent a wide range of foreign language com- 
petence. The FSl-MLA equivalents thus established disclosed that some of the tests, in 
some languages., fail to measure adequately the upper levels of foreign language compet- 
ence, l.e., those levels approaching the competence of the educated native speaker. 

The main study was conducted with the assistance of Educational Testing Service, 
which arranged for the administration of the tests and other instruments after the pro- 
ject staff had drawn the sample of institutions. The sampling of institutions was actu- 
ally done in two phases. From a total of 771 Institutions identified as constituting the 
target population of Institutions, a stratified sample of 192 institutions was drawn and 
invited to participate; it was estimated that these institutions contained a total of ap- 
proximately 6000 students majoring in the five foreign languages Included in the study. 
Because only 146 of these institutions agreed to participate, a second random drawing of 
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101 additional institutions was made, making 293 institutions in all that were invited to 
participate. 

2, Outcomes of the Samplin2 Design 

As of April 9, 1965, when the books were closed on the specification of the sample 
of participating institutions, 203, or 69.3%, of the 293 institutions invited to partici- 
pate had agreed to do so. There was little difference between public and private insti- 
tutions in rate of response, but there was a clear tendency for the larger institutions 
to be more likely to agree to participate. There is no way of knowing to what extent 
this trend may have biased the sample with respect to the criterion variables. 

After the testing, conducted mainly in April and May of 1965, it became evident that 
student response rate fell considerably short of the planned 100% response rate within 
participating institutions. That is, although every effort had been made to persuade 
students at the participating institutions to take the tests, which were given free of 
charge, by no means every student eligible to take the tests did so. It was not possible 
to determine the actual student response rate until nearly a year later, when official 
U. S. Office of Education figures on B. A. degrees earned in 1964-65 became available to 
the project. At that time, it was determined that there were 5427 students (graduating 
seniors majoring in one of the five test languages) available to be tested in the 203 in- 
stitutions that participated in the study. The nctual number of students legitimately 
tested in these institutions (i.e., after 58 underclassmen and 35 graduate students had 
been excluded) was 2784. The overall student response rate, therefore, was 51.4%. The 
tested students represented, in all, 23.9% of the 11,633 students in the target population 
of French, German, Italian, Russian, and Spanish majors present in the class of 1965 in 
the target population of institutions. Detailed studies of student response rates dis- 
closed that (1) there was no significant difference in overall response rate between men 
and women, (2) student response rate was negatively correlated with the size of the in- 
stitution, students in the larger institutio..3 being less likely to volunteer to take the 
tests than students in the smaller institutions, and (3) the stydent response rate was 
higher at private (67.3%) than at public (39.6%) institutions, and this difference was 
consistent for institutions of different sizes. 

The original intention of the study to sample about 50% of the target population of 
students was thus achieved only about half-way; it was frustrated partly by less-than- 
perfect institutional response rates and partly by far-from-perfect student response 
rates. Nevertheless, certain counterbalancing factors produced a sample that was probabty 
fairly representative of the target population. Specifically, the higher institutional 
response rate for the larger institutions tended to counterbalance the lower student re- 
sponse rates in those institutions. (Further comment on sampling biases will be made in 
Section 4 of this chapter.) For this reason there is probably little error in using the 
sample as it stands. 



Though there were highly significant differences in response rat^s over languages, 
these differences appear to be largely associated with Russian, where the response rate 
was consistently low (overall: 31.7%) in both public and private institutions. 

It was inevitable that the data were not complete for the students actually tested. 
Only 2587 students had scores on all four MLA skills tests. Only 2534 students took the 
MLAT. In all, there were 2389 cases complete on the MLA skills tests and the MLAT. 
Questionnaire data were available on all of these, although it was sparse in a handful of 

cases . 

Only students enrolled in teacher preparation programs were supposed to have taken 
the teacher preparation sections (5, 6, and 7) of the MLA tests; there were 1116 students 
who took these additional sections. 980 of these were complete on the four skills tests 
and the MLAT. 

The numbers of "legitimate" cases tested in each of the five languages were as fol- 
lows: French, 1270; German, 413; Italian, 28; Russian, 105; and Spanish, 968. 
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Most of the analyses performed In the study utilized only "regular” cases, l.e., 
students who were apparently native speakers of English who had studied their major for 
elgn language In the course of the normal educational sequence, and had not had special 
opportunities such as protracted residence abroad (more than one year) or special In- 
tensive courses such ar> are available in the Armed Forces. There were 2604 such "reg- 
ular" cases. 



3. Levels of Proficiency Attained by Students in the Tested Sample 

Frequency distributions of scores on the MLA skills tests had a tendency. In many 
cases, to have a negative skew. These findings, coupled with evidence from the FSI equi- 
valences and from studies of possible curvilinear relations among the tests, suggested 
that some of the MLA tests — especially the Listening tests — have "ceilings" whereby they 
do not adequately measure the upper levels of foreign language competence. For example. 

In terms of the FSI equivalents established for the MLA tests, the Listening test meas- 
ures only up to S-3+ in French and Russian, S-4 in Spanish, and S-4+ in German. It may 
be concluded that although the MLA tests are billed as "advanced" tests. In many cases 
they are not sufficiently difficult to discriminate among really advanced levels of 
proficiency . 

Nevertheless, the scores obtained ranged from scores in the chance range to scores 
at or near the maximum possible scores. A general characteristic of the tested samples 
is that they were much poorer In Listening and Speaking skills than they were In Reading 
and Writing. Typically, the "regular" cases had mean scores In Listening and Speaking 
that correspond to FSI ratings of S-2 or S-2+, l.e., in the range of "limited working 
proficiency." In Reading and Writing, however, the tested students tended to have mean 
scores that correspond approximately to an FSI rating of R-3, "Minimum professional pro- 
ficiency." There was some variation across languages in the average FSI equivalents 
attained; French samples appeared to be particularly poor in Listening and Speaking, 
while German samples did somewhat better in these skills, on the average. 

Intercorrelations among the tests were consistent with those previously obtained for 
samples of NDEA Institute teachers after account is taken of the fact that the CELT (Col- 
lege Foreign Language Testing) samples are generally more homogeneous in proficiency 
xevels than the NDEA teachers. The intercorrelations were generally high and suggested 
jhat a large proportion of the variance of each separate skill test is attributable to 
a common factor of overall competence in the language tested. The estimated proportions 
attributable to a common factor were typically between .80 and .90 after correction for 
unreliability. Nevertheless, it appeared that a small but significant proportion of the 
variance of each test measured some specific aspect of foreign language skill. The 
Speaking test generally had the largest proportion of specific variance. 

In the main, the regressions of the tests on one another did not depart significant- 
ly from linearity. Where they did, the most likely explanation is that one of the tests 
has a "floor" and/or a "ceiling" such that the discriminative power of the test is inade- 
quate at either the lower or the upper end of the scale of measurement, or both. 

4. Estimates of Lavels of Foreign Language Proficiency 
Attained in the National Population 

An attempt was made to estimate the amount of bias in the sample with respect to the 
criterion variables introduced by iustltutlon and student response rates that were smaller 
than desired. 

The possible effect of student non-response (l.e., failure of an eligible student to 
volunteer to take the tests) was investigated by a special study based on data obtained 
from 15 medium-large institutions that had tested anywhere from 25 to 90 per cent of 
their eligible students. B'rom transcripts supplied by the registrars, foreign language 
grade-point-averages were computed for all eligible students. It was found that there 
was a statistically significant trend for students who participated in the testing to 
have slightly higher grade-point -averages than students who did not. Through certain 
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statistical procedures, ®?Jg”®ggj’gcMes*TO shifted do«nwards**o^ly a very 

in the testing, the mean MU skills ihraverage. It was cohkuded that 

“Senrn^-^e^^sf ^dfre^SI^ little diKe^ence in the -„atf onal norms-nSt enough 

to Justify any general correction in the norms. 

implicit in the sample ^(^^bufji^'prS 

ents might vary with mdeed occur. Students at the larger institutions 

sis disclosed that such variation did J* . , institutions, and students at private 

tended to score higher th® at public instl- 

®rions!'°F:r's:re^®f:iage'^oups (Ge»®^d^^^^ 

r^urnredirs^rf - 

knowledge of institution size and type. 

, fVio /H -f f prences iii mc'an score between large 

Further analysis disclosed that som ouijiic institutions could be accounted for 

and small institutions or t»«tween Private ^d p^bli ^i ^ti^^ foreign language early 

by the fact that students who * u urere more likely to attend large and/or priv- 

<Le., in elementary school or ^IflaSr! pr^t^ institutioL were more 

ate institutions. Furthermore „ students at ^he larger ,p^^^ language study and the 

likely to have had travel or ^ ^ that were shown to be both very strongly associated 
amount of time abroad orofi^ency. particularly the latter vari- 

with the attainment of high foreign ang 8 P statistically removed from the dif- 
able. Even after the effects remained appre- 

ferences noted with respect o s institutions of different sizes and types. Evi- 
ciable variation among mean scores differences could be explained to a large 

dence was assembled that sugges ed nrlvate institutions are more likely to have 

extent by the fact that the larger P^^hf classroora^ offer more advanced in- 
instructors who use ^f®e^ferger institutions, and to some extent the private 

struction. It was concluded rhe smaller and/or public ones, even after 

rd^nfbX^^^^^ ^ey appear to have better facilities for 

doing so. 

Regional differences in me® scores ar4 more’°p^uiIr!^‘'irSe 

test scores were higher in ^ ^ 4 « annear to differ in their mean overall per- 

s®e time, students / ’'por°ex®ple , students in New England institutions are 

formance, regardless of language. * differences, it should be pointed out, are 

generally high in all languages. ® , variables that are associated with test 

probably related in a f ^o®t of time abroad; no at- 

:“^"LrLS“lie’tfrtu°dy Uional*differences adjusted for the effects of these other 
variables . 

prited with reference to the Fo«i® Service Institute^proW^^^ 

been established, yielded ™ States majoring in foreign languages. It 

be attained by college seniors in the ^ni students atLin relatively low levels of 

was striking that according to * ^ate with a foreign language major can 

skill in spoken forelp l“*f“8es . The “®«i“^8rad®tyith 

speak ®d c®prehend his oroficiency" ®d a "minim® professional 

tLt is, somewhere between a „„ps, the French group show- 

proficiency." This is genera y ^ skills with the written language (Reading 

ing particularly inferior performance. Bu attain median scores that correspond 

®d writing) the Fre®h, is, reading ®d writing skills a "minim® 

approximately “ groups show 1® average performance even in '®|dtag 

rd'^itinf sriUsfro^;er.Tt excluded that the net showing of college foreign 
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language majors in language skills is far from impressive. On the other hand, there are 
appreciable numbers of students who exhibit quite superior competence. Furthermore, the 
college senior populations exhibited clear higher mean scores in foreign language 
skilll (except Speaking, in some cases) than did smples of foreign 
NDEA Language Institutes, and were more homogeneous than the NDEA Institute teacher 

groups . 

When evaluated against the standards of "teacher qualification" established by Myers 
and Melton (1964), the college senior samples show up rather poorly. The Myers-Melton 
standards are very demanding, it would appear. For example, in French, it is estimated 
that only 16% of the national population qualify as 'Superior in Listening, less 
1% in Speaking, 25% in Reading and about 8% in Writing. But relatively few of the CFLT 
population would fall into the Unsatisfactory category by the Myers-Melton standards. 

Still whether one uses the FSI ratings or the Myers-Melton ratings as standards, 
the estimated average level of proficiency of the college senior foreign l^guage major 
is not as high as might be desired. These individuals are particularly deficient in 
audiolingual skills. Only about a third to a half are at a satisfactory level even in 

Reading and Writing skills. 



5, Student Factors in Foreign Language Skill Attainment 

Studies were made to determine what characteristics of the students might be associ- 
ated with their foreign language proficiency. All these analyses involved only the stud- 
ents actually tested, or appropriate subgroups thereof. 

The majority of foreign language majors are women. In the samples studied, °f 
the French majors were women. The figures for the other languages were: German, 59%; 

Russian, 62%; and Spanish, 75%. There were no significant and consistent mean differ- 
ences in tested language proficiency between the sexes (only one difference being stat- 
istically significant) . 

Over the whole CFLT sample, 88.8% were at age 20, 21, or 22; ages of the remainder 
raneed uo to about 50. There was a slight tendency for younger graduates in French ana 
Spanish to have superior scores; this may be a reflection of the fact that more of these 
had had early exposure to these languages in elementary school programs. There was no 
strong tendency for older students (23 and over) to obtain inferior scores. 



The CFLT groups were on the average quite superior in their language aptitude as 
tested by the Modem Language Aptitude Te^ . There were significant differences across 
languages: the language groups rank in the order Russian, French, German, Spanish. 

Language aptitude scores show quite small— barely significant-correlations with foreign 
language proficiency test scores for the complete samples. Further analysis showed, 
however, that language aptitude is a relatively important variable in language profici- 
ency attainment when certain- other influential variables are controlled. 

About 65% of the CFLT sample planned to go into the teaching of foreign languages at 
some level, a slightly higher proportion of women (66%) having such plans than men (61%, y . 
By far the largest number of both males and females who plan to teach expect to do so at 
the secondary school level. Teaching at the college level attracts about 32% of them- 
more men than women. Teaching at the elementary school level is included in the plans of 
only 12% of those intending to teach. Of the complete CFLT sample, about 37% hoped to 
use their foreign language skills in employment opportunities other than teaching. Some 
type of governmental work attracts the largest proportion of these latter, with business 

activity running a close second. 

Reasons for majoring in foreign languages also include various "fringe benefits," 
such as the opportunity to use FL knowledge in travel or study abroad, the serving of 
interests in foreign cultures and literatures, or the promotion of intercultural under- 
standing . 
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Almost universally, students rated all four language skills (listening, speaking, 
reading, and writing) as of "great importance." Speaking and listening, however, had 
ligher average ratings than reading and writing. These findings appear to give strong 
lupport to contemporary policies in foreign language teaching whereby listening and 
tpeaklng are accorded emphasis and attention at least equal to that of reading and writ- 
.ng. Students regard reading and writing as "advanced" skills of great importance, par- 
:icularly for specialized work in foreign languages, but listening and speaking are re- 
;arded as being of even greater importance, .particularly for teaching at elementary and 
lecondary school levels. French, German, and Spanish majors attach relatively more im- 

t ortance to speaking and listening than do Russian majors. It was shown, further, that 
he relative amount of importance a student attaches to listening and speaking as con- 
rasted with reading and writing is to some extent related to his own relative compet- 
ncles in these skills. 

Vocational plans are to some extent associated with mean levels of language profici- 
ency attained. Those planning to teach at the secondary school level are significantly 
inferior, on the average, to the remainder of the group. Those planning to teach at the 
college level are strikingly superior to others. 

Six largely independent ways in which foreign language majors vary are as follows: 

(1) whether, on the one hand, they plan to teach, at some level, or, on the other, use 
their foreign language skills in some kind of employment (usually, government car busi- 
ness) other than teaching; (2) if they plan to teach, whether they plan to teach at the 
elementary or secondary school level; (3) if they plan to teach, whether they plan to 
teach at the college level; (4) whether they have traveled or studied abroad; (5) whether 
they are attracted by the intrinsic benefits of foreign language study (e.g. learning 
about foreign cultures and literatures); and (6) whether they have tended to take other 
languages besides their major language, especially Latin and/or Greek. 

6. Experience and Training Factors in FL Skill Attainment 

One of the major variables found to be associated with foreign language skill at- 
tainment is the educational level at which the student began study of his major foreign 
language. About 64% of the sample started in secondary school. However, on the average, 
students who started in the elementary school were distinctly superior, at graduation 
from college, to students who started in secondary school, and these in turn were (in 
most comparisons) distinctly superior to those who started in college. The simplest ex- 
planation of this finding is that it is due to differences among the groups in the amount 
of time they spent in study; there was no evidence that those who started early had any 
special advantage because of their youth. Starting early did, of course, have the ad- 
vantage that the student had more time to acquire his skill by the time of college gradu- 
ation. Time of beginning foreign language study was largely Independent of language 
aptitude; those who started early did not in general have significantly higher language 
aptitude scores than those who started later. 

Further analyses In which the numbers of foreign language courses taken at the vari- 
i ous educational levels were studied in relation to levels of skill attained only con- 
firmed the conclusions mentioned above: that the level of skill attained is to a con- 

siderable extent a function of the amount of time devoted to foreign language study. 
However, in some cases there was a suggestion that the taking of college language courses 
(as opposed to literature courses) by students arriving from high school with insuffici- 
ent preparation had either no significant effect or even a negative effect on the 
students. 

It was very difficult to obtain reliable information on the kind and quality of in- 
struction that students had had, but there was distinct evidence that a possible favor- 
able influence is the extent to which the teacher uses the foreign language in the class- 
room and requires the student to use it. Use of a language laboratory did not show any 
significant effect, nor did the extent to which the teacher was reported to have a native 
or near-native accent in the language. The major variable in instruction, then, is the 
I amount of time devoted to it, or more precisely, the amount of time during which the 
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student is involved in the use of the foreign language. 

Probably the most important variable associated with the attainment of high foreign 
language proficiency discovered in this study was the amount of time the student had 
spent abroad in a country where his major language was spoken. Substantial numbers of 
students in th€i sample had had such experience. In MLA skills test scores, there were 
highly significant differences favoring groups who had been abroad, particularly those 
who had been abroad for as much as a year of study. The effect of time spent abroad on 
test scores was found to be Independent of the effect of time of beginning foreign lang- 
uage study, even though it was also found that those who started language study early 
were more likely to have been abroad. Amount of time spent abroad was also independent , 
of language aptitude. Although it was impossible to determine whether time spent abroad 
had a causal influence on language proficiency, there was a clear suggestion that travel 
and study abroad improve foreign language skill. 

Another powerful variable, for the relatively few students for whom it is relevant, 
is the extent to which the foreign language is used in the student’s home. Students who 
claim that there is "frequent" use of the foreign language at home perform almost as 
well, on the average, as native speakers, and students who report "occasional" use at 
home tend to do better than students who report no such use. 

There were also indications that various other experiences, such as the taking of 
informal courses, the use of the foreign language in a job situation, or extensive read- 
ing, were associated with higher language proficiency test scores, but it was not clear 
whether such experiences should be regarded as causes or effects. 

The study was not designed to determine whether the study of other modem languages 
besides the major language had any special effect on proficiency in the major language. 
Trends in the data suggested that these effects were mom often positive than negative, 
but the positive effects were not necessarily associated with the degree to which the 
languages themselves were related. It seems likely that students with greater experience 
in the study of other languages do better simply because they have greater interest in, 
and aptitude for, foreign language study. There was little evidence to indicate what 
effect the study of Latin or Ancient Greek might have on proficiency in the student’s 
major modem foreign language. In some comparisons, those who had had Latin in high 
school tended to have slightly lower Speaking test scores, and those who had had Latin 
in college tended to have slightly higher Writing test scores. 

7. Comparisons of Influences of Student and Experiential Variables 

Multiple regression techniques were utilized to assess the independent Influences of 
various student and experiential variables on MLA skills test scores. Multiple correla- 
tions using such variables in predicting MLA skills test scores centered around .60 but 
ranged as high as .73 for some groups. Thus, a very considerable proportion of the vari- 
ation in MLA skills test scores can be accounted for as being associated with student 
characteristics and backgrounds. Among all the variables used in this analysis, amount 
of time spent abroad was generally the most potent predictor, particularly in French, 
German, and Spanish groups. Only of secondary importance were variables such as language 
aptitude, amount of formal training, use of the foreign language at home, and certain 
specific instructional variables such as the extent to which the student had been exposed 
to teachers who used the foreign language in the classroom. 

The superiority of a student in reading and writing, as opposed to his level of 
skill in listening and speaking, was found to be predictable chiefly from one of the 
language aptitude subtests that measures "grammatical sensitivity," and also from the 
extent to which he has not been abroad. (Conversely, the superiority of a student in 
listening and speaking over his level of skills in reading and writing is predictable 
from relatively lower scores on the grammatical sensitivity measure and relatively more 
time spent abroad.) 
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A student’s superiority in productive skills (speaking and writing) as opposed to 
receptive skills (listening and reading) can also be predicted with some reliability from 
background variables such as time of beginning foreign language study, amount of time 
abroad, and use of the foreign language at home. It is interesting that the patterns of 
relationships differed from language to language; this may mean that they are affected 
by the ways in which the languages are typically taught— French and Spanish with more 
emphasis on productive skills and German and Russian with more emphasis on receptive 
skills . 

It was shown that all these patterns of relationships whereby performance on skills 
tests can be predicted from background variables tend to hold over various classifica- 
tions of institutions as to size and type. For example, amount of time spent abroad is 
universally a good predictor regardless of whether the student attended a small or a 
large institution, or a private or a public institution. 

8. Foreign Language Department Characteristics in 
Relation to Student Achievement 

There was significant variation among institutions in the mean scores of their stud- 
ents on the MLA skills tests. To a large extent, these variations vjere a; 3 Sociated with 
institution size, the larger institutions tending to have the students with the higher 
mean scores. It was difficult to identify the specific ways in which departments varied, 
beyond gross characteristics such as department size and the number of languages included 
within a department. Department chairmen’s reports generally failed to agree with stud- 
ent reports on instructional practices. One variable that tended to have some small 
degree of significant relationship to mean student performance was the degree to which 
(according to either students’ or chairmen’s reports) the use of English was avoided in 
Introductory language courses. Other variables, such as whether English was avoided in 
more advanced courses, whether there was an emphasis on audiolingual skills, how many 
courses were offered, or how many degree requirements there were, rarely showed any sig- 
nificant relationships with mean student performance. Our analyses of departmental 
characteristics failed for the most part to disclose the essential differences between 
"good" and "poor" language departments . 

9. Results from the MLA Teacher Preparation Tests Administered to 
Students Enrolled in Teacher Preparation Programs 

Subtests 5, 6, and 7 of the MLA Proficiency Tests presumably measure competencies in 
certain areas related to foreign language pedagogy: respectively. Applied Linguistics, 

Culture and Civilization, and Professional Preparation. These subtests were administered 
only to students enrolled in programs designed specifically to prepare foreign language 
teachers for the public schools. Of the 2874 students who took any of the MLA tests, 

1116 (38.8%) took one or more of the Teacher Preparation tests. It is possible that an 
abnormally high proportion of teacher preparation candidates volunteered to take the 
tests in view of the advantages they gained thereby, but no information was developed as 
to this possibility. In any case, the students who took the teacher preparation tests 
tended to be somewhat inferior to the other students in MLA skills test performance. 

This finding agrees with the finding noted elsewhere that those planning to teach in 
public schools (as contrasted with those expecting to teach at the college level, or not 
planning to teach at all) tended to have slightly lower mean scores on the skills tests. 
Nevertheless, they had mean scores on the teacher preparation tests that were typically 
midway between the pretest and posttest means of teachers enrolled in NDEA Language In- 
stitutes, and they were more homogeneous than those teachers. 

Correlations among the teacher preparation tests and between these and the MLA 
skills tests were interpreted as showing that the teacher preparation tests measure 
quite distinct aspects of competence for teaching and are only incidentally related to 
the student’s overall level of proficiency in pure language skills. 
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Time of beginning foreign language study showed no dependable relation to perform- 
ance on the teacher preparation tests; it was concluded that students probably acquire 
competence in these aspects of teacher preparation principally at the college level. 

Performance on the Applied Linguistics test was most closely associated with scores 
on foreign language aptitude, especially with the scores on the test of grammatical sens- 
itivity. Performance on the Culture and Civilization test was most closely associated 
with amount of time spent abroad and with number of college literature courses taken. 

No variables could be identified that showed any dependable relationships with perform- 
ance on the Professional Preparation test. 

10. Recommendations for the Improvement of the 
Education of Foreign Language Majors 

It must be kept in mind that this report concerns a purely descriptive survey of 
foreign language majors and cannot provide positive evidence as to the causal factors 
that make for superior foreign language proficiency or teaching competence. Therefore, 
it cannot make "sure-fire" recommendations concerning the improvement of the training of 
foreign language majors. It can only point to factors that exhibit strong relationships 
to proficiency attainment and suggest that some of these factors may have causal influ- 
ences. 

The study found that the foreign language proficiency of the J:yp Ica3r gfa3uat ing sen- 
ior majoring in a foreign language was considerably poorer, esrfScially in listening and 
speaking skills, than what might be desired or expected. Nevertheless, it was possible 
to identify groups of students who had.-^"tained satisfactory or even superior levels of 
proficiency. These groups were likely to have the following characteristics: 

(a) They started study of their major foreign language at an early stage, sometimes 
in the gleap^htary school, but at least as early as secondary school, and then continued 
their study through the subsequent years. That is, they devoted a large amount of time 
to formal study of the foreign language. 

(b) They took many courses in literature at the college level. 

(c) At either the high school or the college level, or both, they had teachers who 
made much use of the foreign language in conducting classes and required their students 
to use it. 

(d) They had traveled and studied abroad in a country where their major foreign 
language was regularly used as the medium of communication. (The data suggested, indeed, 
that travel and study abroad was the most important single influence in enhancing foreign 
language proficiency, and that a year’s travel and study abroad had much more value than 
merely a summer tour. Further, it had a stronger influence than starting study of the 
foreign language at an early age. ) 

(e) They had various other experiences that encouraged use of the foreign language, 
such as the taking of informal courses outside regular school work, being employed in a 
job situation requiring use of the language, or voluntarily devoting much time to the 
reading of books in the foreign language. 

If it is desired to improve the average proficiency of foreign language majors, it 
would seem advisable to take steps such that more students will have the advantages ap- 
parently enjoyed by those who exhibited high levels of skill. That is, more young people 
should be encouraged to begin study of a foreign language at an early stage of their edu- 
cation and to continue its study through the college years, not only through formal 
courses but also, and most emphatically, through such experiences as travel and study 
abroad. At the same time, the teaching of foreign languages should be improved, by 
strengthening high school and college foreign language departments (especially those at 
the smaller institutions) so that more courses will be offered and so that more use will 
be made of foreign languages in the classroom. Since foreign language aptitude proved to 
be of some significance in the attainment of certain foreign language skills, consider- 
ation should be given to the greater use of foreign language aptitude tests in guiding 
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students into foreign language study, especially if they plan to become foreign language 
teachers . 



On the basis of findings of the study, one can make no special recommendations con- 
cerning teacher preparation programs other than those already made. Obviously, prospect- 
ive foreign language teachers must have high levels of proficiency, and all the recom- 
mendations vfe have made concerning the improvement of foreign language skills apply with 
equal or greater force to teaching candidates. Superior foreign language aptitude seems 
to be particularly relevant to the attainment of proficiency in applied linguistics, and 
both courses in foreign language literature, and travel and study abroad, are especially 
pertinent to the attainment of the requisite knowledge of foreign cultures and civiliza- 
tions. Any methods that might be found to enhance foreign language aptitude, particular- 
ly in the student’s awareness of the grammatical aspects of language in general, might be 
useful in the preparation of teaching candidates. If the skills and knowledges measured 
by the MLA Professional Preparation test contribute to teaching success, efforts should 
be made to Improve the teaching of those skills and knowledges; the study was unable to 
find any special factors in student backgrounds associated with performance on this test. 



11 - pg/s<>myiiwgnaa^ for Further Studies of 
The MLA Proficiency Tests 



Foreign Language Education 



The study disclosed that many of the MLA Proficiency Tests failed to have enough 
range of difficulty to measure truly advanced levels of skill. The Listening tests were 
especially defective in this regard, even though the research data suggested that they 
were the best overall measures of foreign language proficiency. There were suggestions 
in the data that the Speaking tests are insufficiently reliable and too subject to vari- 
ations in rating standards and/or the examinee’s set or attitude toward the test to serve 
as really accurate Pleasures of speaking fluency. In some cases the Reading and Writing 
tests were not of sufficient difficulty. It would be desirable to improve the MLA Pro- 
ficiency tests both for general use and for use in any further research studies similar 
to the one just now completed. 



It it* definitely recommended that further studies be done of the equivalence of the 
MLA Proficiency Tests to FSI-type "absolute language proficiency ratings," in as much as 
the equivalences established in the present study were based on samples that were not as 
large as might be desired. Samples of at least 100 cases, rectangularly distributed with 
respect to levels of FSI ratings, should be employed, and it might be advisable to delay 
these equivalence studies until the measurement characteristics of the MLA tests have 
been Improved. 



Problems of Sampling and Test Administration 

Further studies along the lines of the present one, whether in foreign language edu- 
cation or in other curriculum areas, should give greater attention to problems of sampl- 
ing, particularly with a view to obtaining better institutional and student response 
rates. It is probable that such studies should utilize: 

(a) somewhat smaller and more carefully selected samples; 

(b) a longer "lead time" in order to prepare institutions to participate more ef- 
fectively; 

(c) better publicity for the study. 

The two-stage stratified optimal sampling design used in the present study could be again 
employed, but its success depends upon securing more adequate institutional and student 
response rates. A study in which only 25% of the population of foreign language majors 
would be tested would be entirely adequate provided that response rates approached those 
planned in the design. Those conducting the study could work through professional asso- 
ciations (such as the American Council on Education, the Modem Language Association, and 
the new American Council of Foreign Language Teachers) in order to secure the desired 
institutional response rates, and foreign language departments should have more adequate 
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plannlng time, clrcv^late better publicity about the purposes of the study, and work more 
closely with students in order to insure that nearly every eligible student takes the 
tests and other research instruments. It would be advisable to plan to give the tests 
somewhat earlier in the senior year than was done in the present study — perhaps as early 
as October or November; It is unlikely that additional learning after that time would 
significantly affect the test results. 

Questionnaire Desii gn 

In the course of analyzing the data from the questionnaires completed by students 
and by department chairmen, it became evident that certain questions were unnecessarily 
ambiguous, that some questions were probably of little Importance, and that insufficient 
information was obtained on certain matters.- A careful perusal of this report will sug- 

•*K‘ ^ 

gest many ways In which the questionnaire design could be improved in future studies. 

One egregious omission from the student questionnaire was that of questions specif- 
ically pointed towards finding out whether the student was a native speaker of English 
or of some other language — perhaps the language in which he was majoring, and if he was 
a native speaker of the major language, whether he had maintained an approximation to 
native speaker proficiency throughout his life. 

Also, the questionnaire should include items that would disclose more precisely tha 
sequence in which the student had various foreign language experiences. If possible, it 
should ask not only exactly how early the student began his major foreign language, but 
also how early he may have begun the study of other languages, and how long he had stud- 
ied them. It should ask how much language study the student had had and how proficient 
the student estimated himself to have been prior to periods of travel and study abroad. 
The answers to questions about the history of the student's formal training in the major 
language should be made more easily codable than was the case in the present study. 
Questions on the student's plans as to teaching and other uses of his foreign language 
should be more clearly separated from questions about "reasons" for majoring in a foreign 
language . 

Experimental Design 

Although further descriptive, evaluative studies like the present one would be use- 
ful, particularly for the purpose of long-range cross-sectional comparisons over a period 
of years, many questions concerning the education and training of foreign language stud- 
ents would hav€>. to be approached through experimental or longitudinal studies In which 
the effects of various types of learning experiences would be assessed by comparing pre- 
test and post test performances. For example, we need to know more precisely what the 
effects of travel and study abroad are, and what level of preparation the student should 
have for him to obtain maximal advantage from such study abroad. Better studies of the 
effects of early language training are also needed. Such information could materially 
aid in the better design of foreign language educational programs. 
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Appendlx A 

\ ' * 

RESULTS FROM THE NEW YORK STATE PILOT STUDY 



Phase I of this study consisted of a pilot investigation in which an attempt was 
made to administctr the MLA Proficiency Tests, the short form of the MLAT, and a prelim- 
inary form of the student questionnaire to a 100% sample of the foreign language majors 
in French, German, Italian, Russian, and Spanish graduating in the class of 1964 from. all 
institutions in the state of New York offering bachelor's degree programs in those lang- 
uages. A preliminary form of the Questionnaire for Chairmen of Foreign Language Depart- 
ments was also tried out. The purpose was primarily to develop experience that would be 
useful In formulating procedures for the national study, and secondarily to obtain norm- 
ative data for the students in the New York state sample. A preliminary report of the 
Phase I study (Carroll, 1965) presented results on response rates, distributions of test 
scores, and recommendations for the conduct of Phase II of the study (i.e., the national 
study with which the present report is concerned) , and promised that detailed analyses 
would be included in the final report on Phase 11. This appendix presents a recapitula- 
tion of the findings previously reported and detailed analyses of Phase I data that are 
believed to be of Interest and pertinence. 

Sample design and response rates 

The statistical universe was defined as consisting of all students enrolled In the 
senior class graduating in 1964 in every institution of higher learning in the State of 
New York listed by the New York State Education Department as having in the academic year 
1963-64 a bachelor's degree program in any of the five languages to be tested (French, 
German, Italian, Russian, and Spanish). (In addition, the Phase I study attempted to 
utilize the population consisting of all senior-class students. Class of 1964, enrolled 
as concentrating in any one of these languages In Harvard and Radcllffe Colleges. Only 
4 out of 51 students in this latter population volunteered to take the tests, and because 
of the low student response rate the results will be ignored.) 

The New York State Education Department identified 70 institutions in the population. 
Of these, 44 (63.0%) agreed to participate in the study, 5 (7.1%) had no students to be 
tested, 1 (1.4%) initially indicated willingness to participate but had to withdraw for 
reasons beyond Its control, and 20 (28.5%) declined to participate or failed to reply. 
Including the institutions that had no students to test, 49 institutions may be said to 

have participated, representing an overall response rate of 70.1%. This figure, inci- 

dentally, is very close to that reported in Chapter III for all institutions in the nati- 
onal study, 69.3%, but the national figure does not include institutions that had no 

students to test. (In the national study, the response rate of New York state Instltu- 

ions was 77.4%, a figure that may reflect the fact that New York State has had a keen 
interest in the MLA teacher certification program and also the fact that in the national 
study, the New York state institutions not only had acquired experience in participating 
In our studies but also had somewhat more time for planning their participation, since 
they received Invitations to participate earlier in the year than they did in the pilot 
study. ) 

For purposes of comparison with the national study. Table A.l has been constructed 
to parallel Table 3.1; it includes only institutions listed in U. S. Office of Education 
tabulations for 1963-64. On this basis, the Institutional response rate for the pilot 
study was only 62.1% as compared to 69.3% for the national study. The difference may be 
attributable to the fact that the national study provided institutions more lead time for 
planning their participation than did the pilot study, in which invitations t^ participate 
were not sent out until late in March (March 23, 1964). Otherwise, the data for the New 
York pilot study are not inconsistent with those for the national study. There is the 
same tendency for larger institutions to be more likely to participate, although this 
tendency is not statistically significant for the New York data (Chi-square = 6.5, 4 d. 
f . , p ^ • 05) . 
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Table A.l 



Institutional Response Rates, New York Pilot Study (1963-64), 
with Comparative Data from the National Study 



Stratum 


Size^ 


Solicited 
Final Sample 
( * Population) 

N^ 


n3 


Participating 

% of Sample 
(Response Rates) 


Response Rates, 
National Study 


5 


40+ 


5 


PUBLIC 

2 


40.0 


80.0 


4 


20-39 


2 


1 


50.0 


68.4 


3 


10-19 


0 


0 


— 


70.8 


2 


5-9 


0 


0 


— 


55.0 


1 


0-4 


2 


2 


100.0 


64.7 


Total 




VO 1 


5 


55.6 


68.6 


5 


40+ 


5 


PRIVATE 

4 


80.0 


90.9 


4 


20-39 


11 


9 


81.8 


86.7 


3 


10-19 


16 


12 


75.0 


69.6 


2 


5-9 


13 


8 


61.5 


70.0 


1 


0-4 


12 


3 


25.0 


56.9 


Total 




57 


36 


63.2 


69.7 


5 


40+ 


10 


TOTAL 

6 


60.0 


83.3 


4 


20-39 


13 


10 


76.9 


79.6 


3 


10-19 


16 


12 


75.0 


70.0 


2 


5-9 


13 


8 


61.5 


65.1 


1 


0-4 


14 


5 


35.7 


58.7 


Total 




66 


41 


62.1 


69,3 



^”Size°' of institution is defined in terms of the total number of FL earned degrees 
reported at the institution, according to U. S. Office of Education tabulations. 

^Includes only institutions listed in U. S. Office of Education tabulations. 



^Does not include 3 institutions that participated in the pilot study but were not 
listed in U. S. Office of Education tabulations. 
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The overall student response rate within institutions was given as 76*2%; this fig- 
ure is based on the fact that the total number of students tested in the 44 participating 
institutions was 667 out of 875 individuals reported by department chairmen as being 
available for testing. The 667 tested students comprised 46.1% of the 1443 students es- 
timated to be present in the statistical universe in New York State. In the national 
study, the overall student response rate (Table 3.3) was given as 51.3%, and the tested 
Individuals comprised only 24.0% of the statistical universe. The student response rate 
in the New York pilot study was considerably higher than in the national study. The per 
cent of the statistical universe tested in the pilot study was also better than in the 
national study, but this is partly due to the fact that the national study employed a 
two-stage sampling design in which only a limited number of institutions were invited to 
participate. 

The figures on student response rates given in the preceding paragraph were based on 
department chairmen’s reports of the numbers of students available to be tested. When 
official U. S. Office of Education figures on numbers of earned bachelor’s degrees became 
available, it was possible to revise the estimates of student response rates. However, 
b. S. Office of Education tabulations did not cover exactly the same institutions identi- 
fied by the New York State Education Department. Furthermore, there was considerable 
ambiguity as to whether the students actually tested were all actually graduating seniors. 
Table A. 2 has been prepared tc parallel Table 3.3 for the national study, but it was dif- 
ficult to adjust the figures so as to be exactly comparable. (That is, in a few cases it 
would appear that the student response rates were greater than 100%, apparently due to 
the fact that non-seniors were tested.) Table A. 2 nevertheless shows similar trends to 
those exhibited by Table 3.3: student response rate was higher at private institutions, 

and it tended to be higher at the smaller institutions. These trends thus appear to be 
inevitable characteristics of sampling survey studies of this type. 

Levels of attainment on MLA tests, tested students 

Norms based on the New York state pilot study were presented in the Phase I report 
but may be regarded as superseded by the national norms presented in the present report. 
The performance of the New York pilot study groups is summarized, however, in Table A. 3, 
in which the means and standard deviations of all cases complete on the MLA skills test 
scores, and of all teacher preparation cases complete on all MLA tests, are given. (Four 
cases from the Harvard-Radcliffe testing were also included here.) The table also shows 
comparstive data from NDEA Language Institute samples as published by Educational Testing 
Service (Appendix E-3) . We may quote the following generalizations about Table A. 3 from 
the Phase I report: 

”1, In French , pilot study cases were superior to both pretest and posttest NDEA 
norms in all four skills except speaking . where they only barely surpassed the pre- 
test mean. 

2. In German, pilot study cases were very clearly superior to even the NDEA posttest 
norms, especially in reading and writing. 

3. The few Italian cases (N = 10) were for all skills somewhat below even the NDEA 
pretest standards. 

4. In Russian , pilot study cases were superior to both pretest and posttest NDEA 
norms in reading and writing; in listening and speaking they were between the two 
sets of norms. 

5. In Spanish . the pattern of means for the pilot study cases was similar to that in 
French; they were superior to both pretest and post test NDEA norms in all four 
skills except in speaking, where they barely surpassed the pretest mean. 

6. The means for teacher preparation cases were in general very similar to those for 
all cases. Thus, prior to exact statistical tests, we judge the teacher preparation 
cases to be a representative sample of all cases tested." 
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Table A. 3 

Comparison of Pilot Study Samples with NDEA Institute Norms 
of Subtests of the MLA FL Proficiency Tests 



NDEA Norms 



Pilot Study 



Pretest 



Posttest 



All Cases 



Teacher 
Prep . Cases 



FRENCH 

Listening 
Speaking 
Reading 
Writing 
App. Ling. 

Civ. & Cult. 
Prof. Prep.* 

GERMAN 

Listening 
Speaking 
Reading 
Writing 
App. Ling. 

Civ. & Cult. 
Prof. Prep.* 

ITALIAN 

Listening 
Speaking 
Reading 
Writing 
App. Ling. 
Civ. & Cult. 
Prof. Prep.* 

RUSSIAN 

Listening 
Speaking 
Reading 
Writing 
App. Ling. 
Civ. & Cult. 
Prof. Prep.* 

SPANISH 

Listening 
Speaking 
Reading 
Writing 
App. Ling. 
Civ. & Cult. 
Prof. Prep.* 



N 


X 


a 


N 


X 


a 


4720 


38.1 


8.8 


4770 


42.8 


8.6 


4494 


72.1 


19.3 


4577 


81.0 


17.8' 


4722 


43.3 


10.8 


4770 


45.7 . 


10.5 


4723 


43.0 


12.9 


4769 


45.7 


12.6 


4641 


44.9 


8.1 


4612 


51.1 


8.9 


4640 


43.6 


8.5 


4611 


49.3- 


8.7 


10771 


59.5 


8.2 


10909 


67.0 


6.5 



N 



X 



330 44.0 7.6 

330 73.8 12.5 

330 49.3 9.0 

330 47.6 10.1 



1102 

1086 

1103 

1103 

1102 

1101 

10771 



64 

62 

64 

64 

64 

64 

10771 



38.4 

79.5 

44.4 

45.8 

46.8 
46.7 

59.5 



40.0 
88.2 
45.2 

51.1 

46.1 
49.8 
59.5 



9.1 

17.7 

11.9 

16.5 

9.1 

10.0 

8.2 



6.0 

15.4 

10.3 

14.4 
6.5 
8.0 
8.2 



1256 

1232 

1256 

1258 

1179 

1178 

10909 



64 

62 

64 

64 

64 

64 

10909 



42.1 

85.3 

48.1 

48.3 

53.0 

53.4 

67.0 



40.5 

98.0 

47.5 
55.3 
50.7 

54.5 

67.0 



9.1 

19.4 

10.9 

14.6 

9.0 

9.5 

6.5 



5.9 

14.8 

10.7 

14.7 
7.8 

8.5 

6.5 



597 

584 

556 

555 
515 

556 
10771 



4394 

4271 

4446 

4444 

4393 

4342 

10771 



38.4 
73.0 

35.5 

48.6 
42.8 
47.2 
59.5 



39.3 

72.1 

42.2 

46.2 

43.3 
49.6 
59.5 



7.0 

17.7 

10.7 
16.5 

6.8 

9.2 

8.2 



8.0 

20.4 

10.2 

13.7 

7.8 

9.7 

8.2 



587 

570 

545 

545 

505 

545 

10909 



4400 

4316 

4544 

4547 

4494 

4443 

10909 



43.3 

87.1 

38.2 

53.4 
47.6 
51.9 
67.0 



41.8 

78.2 
44.6 

50.2 
49.4 

56.2 
67.0 



7.5 

17.0 

10.8 

14.8 

6.7 

8.4 

6.5 



7.6 

17.5 

9.5 

12.8 

8.5 
8.9 

6.5 



66 


45.0 


8.0 


66 


91.8 


16.4 


66 


53.3 


10.1 


66 


55.4 


14.8 


10 


38.8 


7.2 


10 


80.7 


16.5 


10 


39.8 


9.7 


10 


46.1 


14.4 




— 


— — 








42 


42.6 


6.4 


42 


79.2 


16.6 


42 


43.4 


13.5 


42 


59.8 


11.7 






— 


** 






181 


43.5 


6.9 


181 


74.9 


13.8 


181 


46.7 


9.1 


181 


53.0 


11.1 




— 


— 


mm 













N 


X 


a 


159 


43.5 


7.8 


159 


72.9 


12.8 


159 


48.9 


8.9 


159 


47.4 


9.9 


159 


49.0 


7.6 


159 


48.6 


7.3 


159 


61o0 


7.6 


21 


44.3 


7.0 


21 


87.1 


13.1 


21 


51.6 


9.2 


21 


55.0 


13.4 


21 


48.4 


10.3 


21 


52.9 


8.1 


21 


59.2 


8.8 


6 


37.7 


5.2 


6 


81.8 


15.8 


6 


38.0 


5.0 


6 


44.0 


7.6 


6 


46.2 


5.7 


6 


44.3 


4.3 


6 


53.7 


7.5 


6 


43.8 


5.1 


6 


79.2 


12.1 


6 


44.8 


14.5 


6 


66.3 


6.4 


6 


46.2 


5.1 


6 


50.3 


13.4 


6 


55.0 


8.7 



98 


43.9 


6.4 


98 


76.0 


11.5 


98 


47.2 


8.0 


98 


54.1 


9.9 


98 


45.3 


6.6 


98 


55.7 


8.5 


98 


61.5 


6.8 



*Ihe NDEA institute norms for the professional preparation test, which Is Identical for 
all languages, are not given separately by languages. 
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Concemlng the teacher preparation tests, the Phase I report noted: 

"Means for the teacher preparation tests for the teacher preparation cases in 
the pilot study were in general between the corresponding pretest and posttest means 
in the NDEA norms. The means for Applied Linguistics and Civilization and Culture 
tests were generally higher than those for the Professional Preparation test, and in 
the case of the German, Italian, and Russian pilot study samples, they were somewhat 
below even the NDEA pretest norms." 

It is now possible to compare the New York pilot study data with those assembled in 
the national study. Table A. 4 has been prepared to parallel Table 4.1. By comparing 
these two tables, it may be seen that the New York data are in general quite comparable 
to those from the national study. Critical ratio tests of mean differences disclosed no 
significant differences at the 5% level except for the Spanish sample, where the critical 
ratios for the four MLA skills tests were, respectively, -2.68**, -9.43**, -2.37*, and 
-2.81** (**p < .01, *p < .05, the negative sign indicating that the New York sample mean 
was smaller) . The poorer showing of the New York sample in Spanish may reflect the kind 
of regional difference mentioned in Chapter V, Section 3 where it was found that mean 
scores of regions were somewhat correlated with the popularity of the language. Spanish 
is not as popular in New York state as it is in some other regions, and it is not as pop- 
ular as French or Russian. Furthermore, the national sample may have included more 
native or near-native speakers of Spanish than the New York sample. (Both Tables 4.1 and 
A. 4 are based on both "regular" and "odd" cases, i.e., native speakers and other special 
cases are not excluded.) There was only one instance of a highly significant difference 
in standard deviations, for the French Speaking score, where the New York sample was much 
more variable (F329, 1169 “ 1»71, p < .01). This could possibly have been due to greater 
variation in rating standards at the time the New York group tapes were evaluated. The 
comparative values for the skewnesses of the score distributions are not inconsistent be- 
tween the two samples. 

Because the New York pilot study samples were in general quite similar to the nation- 
al study samples, it was not thought worthwhile to do numerous detailed analyses of the 
New York samples. In any event, the small numbers of cases available in the New York 
samples would often have precluded such analyses. The remainder of this Appendix A is 
devoted to the reporting of a number of comparative analyses of possible interest. 

Factors Associated with the Attainment of Foreign Language Skills 

Data collected in the New York pilot study did not conveniently permit the breakdown 
of the sample by vocational plans in the same way as the national sample. However, Table 
A. 5, analogous to Table 6.12, suggests that those planning to teach at the elementary 
school level were generally inferior, on the average, to those planning to teach at other 
levels, and that students planning to teach at the college level were generally superior. 
Only the latter finding agrees with one in the national study; in the national study, the 
elementary school teaching aspirants were not significantly different from the secondary 
school aspirants. Tabl.> A. 5 agrees with data from the national study in showing that 
students planning to use foreign language skills in government or business work were re- 
presentative of the total tested group. 

In Table A. 6, there are shown the numbers and percentages of "regular" cases who 
began the study of their major language in elementary school, high school, or college. 
These data are entirely consistent with those presented in Table 7.1, except that if any- 
thing, there are relatively fewer college starters in the New York samples. Table A. 7 
compares the mean scores on the skills tests of these various groups, as did Table 7.2 
for the national study , and shows very much the same trends : that those who started in 

elementary school attained higher scores than those who started in high school, and that 
the latter, in turn, performed better than those who started in college. The trends were 
highly significant in French and Spanish groups. In the German groups, the trend was 
confirmed, although because of small numbers of cases it was not significant; it is note- 
worthy that in the New York samples the directionality of the trend for German was the 
same as for the French and Spanish groups, whereas it was reversed in the national sample. 
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Table A. 4 

Data from MLA Skills Score Distributions, New York Pilot Study 
(All Cases Complete on 4 MLA Skills Tests) 



Statistic 


Listening 


Speaking 


Reading 


Writing 




FRENCH (N = 


330) 






Max. Possible Sccre (Form A) 


56 


125 


70 


80 


Highest Obtained Score 


56 


116 


70 


72 


Median (Estimated) 


45.0717 


72.2625 


49.4224 


47.5869 


Mean 


43.9727 


73.7758 


49.3030 


47.6212 


Lowest Obtained Score 


24 


39 


29 


24 


Chance Score 


29 


— 


32.5 


— 


Standard Deviation 


7.6298 


12.4732 


8.9696 


10.0589 


(Measure of Skewness) 


-0.4321** 


0.3640** 


-0.0399 


0.0102 




GERMAN (N = 


66) 






Max. Possible Score (Form A) 


56 


125 


70 


80 


Highest Obtained Score 


56 


119 


68 


78 


Median (Estimated) 


45.9942 


91.7345 


54.8062 


58.4458 


Mean 


45.0455 


91.7879 


53.2879 


55.4091 


Lowest Obtained Score 


28 


57 


33 


22 


Chance Score 


29 


— 


32.5 


— 


Standard Deviation 


7.9304 


16.3248 


10.0676 


14.7391 


(Measure of Skewness) 


-0.3589 


0.0098 


-0.4524 


-0.6181* 




RUSSIAN (N 


II 






Max. Possible Score (Form A) 


56 


125 


70 


80 


Highest Obtained Score 


55 


121 


69 


79 


Median (Estimated) 


42.2110 


74.5163 


41.4128 


60.7136 


Mean 


42.5‘’14 


79.1667 


43.3571 


59.7857 


Lowest Obtained Score 


33 


48 


23 


37 


Chance Score 


29 


— 


32.5 


— 


A 

Standard Deviation 


6.3401 


16.3924 


13.3467 


11.5995 


g^ (Measure of Skewness) 


0.1706 


0.8511* 


0.4370 


-0.2400 




SPANISH (N 


= 180) 






Max. Possible Score (Form A) 


56 


125 


70 


80 


Highest Obtained Score 


56 


116 


67 


77 


Median (Estimated) 


43.6844 


72.1865 


45.9639 


53.1280 


Mean 


43.4778 


74.7556 


46.9722 


52.9333 


Lowest Obtained Score 


27 


48 


30 


28 


Chance Score 


29 


— 


32.5 


— 


Standard Deviation 


6.8138 


13.6844 


8.3762 


11.0798 


gj^ (Measure of Skewness) 


-0.0910 


0.5632** 


0.3611* 


-0.0527 



**p < .01, *p < .05 for hypothesis of = 0 and normality of distribution. 



Table A. 5 

Means and S.D.'s of MiA Skills Test Scores by Vocational Plans (Sexes Combxned) 

and by Major Language, 

New York Pilot Study Sample 
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Table A. 6 

Numbers and Percentages of "Regular" Cases, by Time of 
Beginning FL Study, by Language (Excluding Italian) 
(All cases complete on four MLA Skills Tests) 

Data from Institutions in the State of New York 



Time of 
Beginning 








Language 








Total 
N % 




French 
N % 


German 
N % 


Russian 
N % 


Spanish 
N % 


"Grade school" 


53 


17.5 


1 


1.8 


0 


0.0 


14 


8.4 


68 


12.7 




High school 


231 


76.2 


31 


54.4 


2 


20.0 


131 


78.9 


395 


73.7 




College 


19 


6.3 


25 


43.9 


8 


80.0 


21 


12.7 


73 


13.6 




Total 


303 




57 




10 




166 




536 








Table A. 7 



Means and S.D.'s of MLA Skills Test Scores for Groups Starting 
FL Study at Different Educational Levels, New York Pilot Study Samples 
"Regular" Cases Complete on MLA Skills Test Scores) 



Time of 

Beginning N 



Grade School 


53 


High School 


231 


College 


19 


Total 


303 



F 



Grade School ^ 


32 


High School j 




College 


25 


Total 


57 


F 





Grade School 
High School 
College 



14 

131 

21 



Listening Speaking Reading Waiting 

X a X a X a X 



FRENCH 



46.11 7.60 

43.34 7.49 

40.74 6.15 


78.42 10.94 
72.24 11.63 
65.95 9.81 


51.40 7.70 
48.77 8.97 
44.21 5.90 


50.02 8.94 
46.91 9.73 
43.74 9.12 


4.52* 


10.01** 


5.03** 


3.62* 




GERMAN 






44.25 8.31 


88.69 17.22 


52.19 10.75 


53.75 15.04 


42.14 6.62 


88.32 12.27 


50.36 7.89 


51.73 12.34 


0.95 


0.01 


0.45 


0.27 


46.79 5.70 

43.14 6.56 

40.38 6.61 


SPANISH 
84.14 9.87 

72.91 12.32 

72.43 12.11 


51.36 7.62 
46.24 7.89 
44.29 7.24 


58.86 9.43 
52.08 10.56 
48.62 9.77 


4.02* 


5.50** 


3.57* 


4.07* 



Total 

F 



176 
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(It will be recalled, however, that for the national sample the discrepancy seemed to be 
explained by the fact that for some reason the German elementary and secondary school 
starters had had less total exposure to the language than those who started in college.) 

Likewise, in the New York samples the amount of time spent abroad manifested the 
same strong association with test scores, as shown in Table A. 8, as it did in the nat- 
ional sample (Table 7.9). (No cross-classification with time of beginning foreign lang- 
uage study was made for the New York study samples because of small numbers of cases.) 



Table A. 8 

Mean Listening Test Scores (and N's) by Time Spent Abroad, 

New York Pilot Study Samples 





French 


German 


Spanish 


Year Abroad 


50.57 (49) 


48.67 


f (9) 


49.80 (20) 


Summer school abroad 


46.77 (48) 




1 (3) 


44.92 (25) 


Toured abroad 


44.40 (45) 


46.10 


(10) 


44.48 (25) 


Never abroad 


40.29 (159) 


40.56 


(32) 


40.86 (96) 


TOTAL 


43.61 (301) 


43.39 


(54) 


43.10 (166) 



Finally, regression analyses of New York pilot study data exhibited patterns of 
behavior in the predictors that were highly similar to those found for the national 
sample. Table A. 9 gives beta-weights and multiple correlations for a set of predictors 
that includes the MLAT total score and several measures of the amount of formal training. 
It is roughly comparable to the regression analyses shown in the second set of data 
presented in Tables 8.2, 8.5, 8.4, and 8.5, except that it does not include Time Abroad. 
Save for this omission, the multiple correlations obtained for the New York study would 
probably have been as great in magnitude as those for the national study. 
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Appendlx B 



PJ-251 

Fora 10- (roviiod) 



Buretu of tho Budflot No« Sl"R4S4«l 
Approval expire! Oeceid>er 31, 1965 



STUDY OF FOREIGN LANGUAGE MAJORS 
IN UNITED STATES COLLEGES AND UNIVERSITIES, 1965 
Conducted by Harvard University under NDEA Title VI 



contract with the U.S. Office of Education 

QUESTIONNAIRE FOR FOREIGN LANGUAGE MAJORS 



TO THE SfUDCNTt Thii questionnaire haa been devised to obtain essential background 
iaforaetion to be analyaed in conjunction with the nationwide testing project in 
which you are participating. 

This testing project is being supported by the United States Office of Education 
in its effort to help iaprove the quality of foreign languag>» instruction in our 
nation's schools, colleges, and universities. Your careful attention to this 
questionnaire will be a service not only to yourself but aliio to future students of 
foreign language in our country. 

Please follow the step-by-step directions closely in order to insure conpleteness 
and accuracy. The use of a pencil is reconaended so that you can correct mistnkes more 

easily. 

Your answers to this questionnaire will be treated in confidence. However, your test 
results will be sent you in Septeaber or October, 1965, at the address you specify below. 



Institution Code 



Please print the two code nunbers given you by 
the test adninistrator in the appropriate boxes: 

PART ONE 



Student Code 



Directions; Please f?M in the blanks by printing clearly or checking the proper answeVr for 
each of t'ue itens below: 



1) Naae 



(last) 



(first) 

Peroanent address to which your test results nay be gwiled: 



(initial r 



(no. ^ 



(street) 



(city) 



(state) 



2) Sex: [.] M [3 F 
1 2 





3) I expect to greduete in (nonth) 
inititution) - 



(year) 



from (naoe of 



1 4) In the boxes below, check the £!!£ language in which you are being tested: 

] FRENCH 
oat ] SPANISH 
OeL ] ITAUAN 
oaC 3 GERMAN 
leC 3 RUSSIAN 

This is the TEST LANGUAGE referred to elsewhere in this questionnaire, 
this language is A language in which you are ''inajoring” or "concentrating. 



It is assumed that 
If you are 



officially "• 
here: 



Bjoring" in two or more languages (do not count minors), print their names 



Is) Write "T" in the box representing your TEST language. In each of the other boxes, '"i'e 
' approximate number of years of itudy you have had in the given Unfluage. 

ichool 2 nd college combined. Report to neareit 1/2 year, JLt ? J - Z? * Write 0 if you 

have taken no courses. Hake sure that you have marked each box. 

ITALIAN 



c 3 FRENCH 


02 [ 3 SPANISH 


ohI 


[ 3 GERMAN 


1,[ 3 RUSSIAN 


I 



OTHERS (specify) 



1 6) Have you ever taken previously any of the MLA Proficiency Tests for Teachers and Advanced 
Students? (These are the tests being given under the present project) 

c 3 NO c 3 YES (If "Yes,*' complete items A and B below): 

* A) Mhich language test(s) did you take? Check the language(s) involved: 

[ 3 FRENCH [ ,3 SPANISH [ ^3 ITALIAN [^^3 GERMAN [^^3 RUSSIAN 

B) Mhich particular^test(s) did you take? If these were in more than one language, explain 

[*JruSTENlNG [23 SPEAKING [ j3 READING [ 43 SITING [ ,3 CIVILIZATION AND CULTURE 
[ §3 APPLIED LINGUISTICS [ ?3 PROFESSIONAL PREPARATION 



PLEASE TURN THE PAGE 
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Appendix B (continued) 



PAGE 2 



This p*oe ii for reference you conplete the tible on peflt 3. 

66 iuro to rood tho Diroctiont on thit pnyo boforo you itirt work on tho chs?t« 



TYPE OF COURSE 

1 - any eourie et grede ichool level 

2 - any high ichool courie 

3 - beginning or introductory college lanojaoe 
courie. A language (ai oppoied to 
literature) courie ii a courie which nay 
involve loae reading, but ii prioarily 
concerned with baiic graanr itudy, 
liitening '^nd ipeaking. 

i - "interaediatt" language eourie at the 
college level 



- "advanced" language course at tho college 

level. Include here eonveriation couriei, 
phonetici eouriei, advanced granaar or 
writing couriei. 

- literature couriei at the collage level. 

A literature couri's ii devoted priaarily 
to the reading of texti in the foreign 
language (not in tranilation), rathar 
than the itudy of the language ai luch. 

- any couriei not covered by the above 



DURATION (leinesters or quarters) 

Use column B or C depending on whether the ichool 
If a high ichool or grade ichool courie, consider 
as 2 leaeiteri. Consider a college luaaer courie 



was a leaeiter or quarter syitea. 
a half year as 1 semester and a full year 
as 1 semester. 



D CUSS HOURS PER WEEK 



Write in the number of classroom hours fist week for this, courie. A single class period of 40 
minutes to 1 hour should be considered an "hour" in adding up the total per week. 



E SIZE OF CUSS 

Mark the estimated number of students there were in this class (under one teacher) according 
to the following icele: 

1 - (fewAr than 10) 2 - (10 to 19) 3 - (20 to 29 ) 4 - (30 or oves) 



F TEACHER'S CUSSROOH UNGUAGE 

1 - the teacher spoke almost entirely in English except when reading foreign sentences or 

covering assigned drills ^ , , . ... 

2 - the te&cher occeiiontlly would diicuii things using the foreign Isnguage, converse with 

students in the foreign language ... 

3 .. during class periods, the teacher spoke the foreign language almost exclucively, using 

English as little as poisible 



G STUDENTS' CUSSROOH UNGUAGE 

1 - speaking in English was the general rule, except for some short periods of conversation 
in the foreign language , . , 

?, - during class periods, the students were required to speak in the foreign language; only 
occasionally would Enpliih be allowed 



H UNGUAGE UBORATORY USE 

1 - the use of a language laboratory was not involved in this course 

2 - a language laboratory was used on an informal or occasional basis 

3 - the use of a language laboratory was an important and integral part of this course 



I TEACHER'S PRONUNCIATION 



1 - the claisroom teacher was not a native speaker of the foreign language and pronounced 

it with a definitely non-native accent 

2 - the classroom teacher had a somewhat non-native accent 

3 - the classroom teacher wai a native speaker of the foreign language, or had a 

"native" accent 



J NATIVE INFORMANTS (native speakers of the language who assist the teacher by conducting 

drills or conversation sessions) 

1 - the students did not meet with native informants in connection with' thii course 

2 - the studenti met with native informanti on a regular basis for lan guage practice 

FINAL EXAMINATION (Listening): If the final exasAnationi in thii course involved (or will 



K 



probably involve i in the case of curren'^ courses) the students listening to 
•tatenen^f or questions ipoken in the foreign language, mark "1. If not, 
mark "0." 



I FINAL EXAMINATION (Speaking): If the final examination! in this courie involved or will 

^ involve speaking to the teacher (or a recorder) in the foreign language, mark 1, 

If not, mark "0/1 



M final GRADE: My grade in thii course was (or probably will be): (If course lasts more than 

1 semester, give avorage grade as closely as possible:) 

0 - Fail 1 - about "0" 2 - about "C" 3 - about "B" 4 - about "A" 

(or equivalent) (or equivalent) (or equivalent) (or equivalent) 
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PAGE 3 



PART TNO - COURSES IN MAJOR LANGUAGE 



Dlrectloni ; In the table below on "Courses in Major Language, the 25 rows * 

would like you to answer a number of questions about each of the formal courses which you have 
taken in your MAJOR language (the language you will be tested on) from grade school up to and 

including the present tia«. 

A formal course is any course offered in a regular school system and for which school credit is 
Given Formal courses may include summer, night or correspondence courses, provided they 
firSihool “edi!“ A^y cLrses which you took "on your own," including courses which you audited, are 
not considered formal courses and should not be included here. 

Space has been provided in the left-hand columns of the table to 
its grade level (for example, 7th grade, 1st semester fresh.), and the INSTRUCTOR S NAME (if knownJ. 

Th* course title can be approximate (e.g.. Begin. French; 10th Cent. Lit., etc.J. 

Wist you write in these first three columns is for your own use to help you remember the courses, 

to your indications may be brief. Please be sure, however, that you Jms y^ are 

caurM* are in chronological order, beginning with the earliest courses and ending with the ones you are 

cS^J ^kinS an Mse two or’ more courses were t. '.en simultaneously, list in any convenient order.) 

After listina the courses, you will be ready to fill in the requested information for the 
the table The answer in each case will be some number, either a number requiring no code (for example, 

nuAer of’classroom hours per week), or a number corresponding to one of the coded answers given in 

the "Answer Key" provided on the facing page. 

^rtLuUr iS%it/^*g(So!;»t kSSw), but a. f^r as possible try. to answer ell the question,^ 



COURSES IN MAJOR LANGUAGE 



GRADE APPROX. COURSE 





Appendix B (continued) 



PART THREE 



PASS 4 



OlMfetlMi; Check the following ite»i •• e pproprUte. 

1 ) nr ahat reiiond) did you decide to coneentme n tile Itngutg. In «»ich »o« «r. iig 

7T^ *.nr: r.r.ig. u.g..g. .. u.g»g... » ». - .«•> « >«•» « 

Mhi«h you plan to teach: 

[ ] at the elenentary achool level 
[ ] at the lecondary achool level 
C ] a college, univeriity, or graduate achool 

I ] other (apecify) — 

[ ] I expect to uae »y foreign language knowledge i« iv future eniploy«ent or profeaaiou. 

^ ao, apecify the kinds of work you are contecq)lating: 

r 1 Linguiitici . ^ % 

[ ] Buaineaa activity A I D etc.) 

[ ] Governaent work (foreign aervice, United Nationa, a.i.d., 

[ J Military aervice 

[ ] Work aa a tranalator or interpreter . -te ) 

[ ] Other profeaaional work (medicine, acience, library work, 

[ ] I expect to uae ray foreign language knowledge in travel " 

[ I ; :b i; ; a; .. .. ...... 

•nd culturttl fiipccti of tho lonfluaflCt 

[ ] I am intereated in intercultural “"J JJ^biect^ 

“ ] No particular reason other than the fact that I needed J J • 

] ether reasoni (specify) 



[ J Other reasons ispeciiy/ — , — 4U^ " 

21 1. ,u« of th. t»» ««;««>»} rimm. 

” . nvM* Imnnrtance 



Listening: 

Speaking: 

Readirq: 

Writing: 



little importance 
iC ] 
iC ] 
iC ] 
iC ] 



sone iaportance 
2 [ ] 

2 [ ] 

2[ ] 

2[ 3 



great importance 

sC ] 

3[ ] 






] 



~ 3) Are yli’planning to qualify for a ll ^se or certificate for teaching in public school.? 



-4) Co^.lderlni flfflk louL asM S'oShS)?' 

audited any course, in that language iwnetne sunoer, u*«ni., k 

[ ] No C ] Yea 



5) What courses have you taken in Latin or ureekV 

[]None [] One or more in high school [, ] One or more in college 

pj, t»i. .....ti... .y. ti..t 

d.ri», ti«. (12... ....... 

are not necessarily school credit courses). 

Have you had such sumoer study abroad? 

courses in your major language during that time. 

Have you had such a school year study abroad? 

■ fl) SIcId'^L qulnill V it you answered SSS. to ^her queition b or 1. n your An.wei^ U EHIB 

;.d 7 ». i. ».» (2. .PP»P.i.t. •» '2. q...tl.n 2.1»1 

Save you tr.v.ll«l .. . wurlM 1. t2e ii.jor-l.ng.sg. ..."try ."./or 2.d > progr.. of ,olf-.tudy 
in the major-language country? 

[J No [J Ye. 

9) DO you and/or your pare nts apeak your major language at nome? 



frequently 



[ ] No [ ] Yea, occas ionally CJ f' 

10) HaveT^H'hJd occasion“tc use your «jor language 

f«L5n friend., or in some other extracurricular aituation? 

L ] No [ ] Yea, to a moderate extent [ J Yea, extensively 

“» »jT;r.;.4.5“ssSk”2"s[ «iS".«“d5“r;K.";s ;4iri’..o... 

C ] I have not done any reading in my major language except for material required by or 
^ involved in course work. 

[ ] I have read, independently of course work, a few pages of material in my major 0 « * 

[ J I 2». r..d, l,d.p.nd..tly .f ...... «-.2. ... •» 2..2. C .2. In PP -J« 

t : lnd.p.ndent»y «f ...... ‘2..« «. “.*• <" *" *» 



END OF QUESTIONNAIRE 
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OE -4224-1 (5-64) 

PJ-251 

Form 11 



Bureau of the Budget No. 51-R454 
Approved: April 8, 1964 
Approval expires: March 1, 1965 



STUDY OF FOREIGN LANGUAGE MAJORS 
IN UNITED STATES COLLEGES AND UNIVERSITIES, 1964 



conducted b^' Harvard University under NDEA Title VI contract with the U.S. Office of Education 



QUESTIONNAIRE FOR CHAIRMEN 
OF FOREIGN LANGUAGE DEPARTMENTS 



TO THE CHAIRMAN: In order to obtain a certain amount of general information relevant to the organi- 
zation and conduct of the foreign language programs in your department, we would greatly appreciate 
your answers to the questionnaire below. In some cases, the appropriate answer will consist simply 
of a number or check mark. In others, it will be necessary to describe at some length certain 
details of your language program. Although a certain amount of writing space has been provided in 
each instance, you may find it necessary to append a separate sheet for an.wers to some questions. 
If this is necessary, please number these questions as they are numbered in the questionnaire. 

We would like tc take this opportunity to thank you in advance for your attention to this 
questionnaire and to express our appreciation for your participation in the project as a whole. 



NAME OF INSTITUTION; 

If this department is at a particular branch of the institution, please indicate: 

OFFICIAL NAME OF DEPARTMENT; 

NAME OF DEPARTMENT CHAIRMAN: 



INSTITUTION CODE 



(do not fill in) 



DEPARTMENT CODE 



(do not fill in) ^ 



Pipasp chftck all the languages for which courses are given in your department. (Check all that apply J 
iC 3 FRENCH 2[ ] SPANISH 4[ ] ITAUAN 8 [ 3 GERMAN 

16[ 3 RUSSIAN 32[ 3 OTHERS (Specify) 



Please turn to page three . 
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Appendix C (continued) 



a 



1) At your institution, when do students normally elect their "major" or subject matter field of 
concentrated study? (Circle one number) 

FIRST YEAR 

If semester system: 1 2 

If quarter system: 12 3 

2) What guidance, if any, is available to students in helping them make decisions as to "major" or 
field of concentrated study? (Check all that are applicable) 

l[ ] None; student makes decision without help from faculty or staff. 

2[ ] Help is available from a faculty advisor assigned to the student. 

3[ ] The student may seek the advice of members of the foreign language department (s). 

4[ ] Assistance is available from a counselling and testing bureau on ihe campus. 

5[ ] Other (specify) ^ ■■ ^ - 

3) What requirements, if any, must undergraduates meet before being accepted as "majors" in your 
department? (If applicable, describe requirements in terms of academic record, scores on 
aptitude and proficiency tests, and the like.) 



4) Please describe the specific foreign laiguage DEGREE requirements for an a.b. degree wiini, 
major in vour department . Include in your answer how many courses of each of several types 
(language, literature, composition, conversation, culture and civilization, etc.) are required. 
Please indicate also any use of proficiency tests to check foreign language achievement. 



5) Does your department offer programs leading to any degree beyond the A.B.? (Check all that are 
applicable) 

0[ ] None. iC ] M.A. 2[ ] Ed.M. 3[ ] Ph.D. 

4[ ] Ed.O. 5-9[ ] Other (Speclfy)_^ 



SECOND YEAR 
1 2 



THIRD YEAR 
1 2 



OTHER (describe) 



Please turn to next page . 



- 230 - 



Appendix C (continued) 



4 

k) Ple.»e l.d&a.e the .-nber of -maJo rf rjS"! T.rgMlcab!iM;- °' 

V lanoiiaae. For languages not covered in your department, write n.a. i noi v, 

co..»,ru CD 4 NTCH ITALIAN GERMAN RUSSIAN INTERDEPARTMENTAL (describe) 



class of 1964 
(seniors) 

class of 1965 
(juniors) 



,nd.ea.e »it. a CecK «* 

the main institution library '»r by your departmental library. icnecK 

1[ ] Georgetown 'Monograph Series in Language and Linguistics 
2[ ] Language (Linguistic Society of America) 

3[ ] Language Learning 
4[ ] Modern Language Journal 
5[ ] PMLA 
6[ 3 None of these 



i', o departmental library 

FRENCH 

llC ] Le Frangais Moderne 
12[ ] French Review 
13[ ] Romance Philology 
14[ ] Romania 
10[ ] None of these 

SPANISH 



ITALIAN 

4l[ ] Italica 

42[ ] Lingua Nostra 

43 [ ] Rendiconti della Reals Accademia d’ltalla 
44 [ ] Studi di Filologia Italiana 
40[ ] None of these 
GERMAN 

81 [ 3 German Life and Letters (London) 

02[ 3 German Quarterly 
03[ 3 Germanic Review 
64[ 3 Monatshefte 
80[ 3 None of these 



2l[ 3 Rispania 

22[ 3 Revista de Fllologfa Espanola 

23C 3 Revista Hispanica Moderna 

24[ 3 Thesaurus - Boletln del Institute 
Caro y Cuervo 

20[ 3 None of these 
RUSSIAN 

l6l[ 3 Revue des Etudes Slaves 

162[ 3 The Slavic and East European Journal (AATSEEL) 

163[ 3 Slavonic and East European Review (London) 

164[ 3 Zeitschrift fttr Slavische Philologie 

I 160[ 3 None of these 

in u. .f the 

in undergraduate foreign language rr^see footnote if your institution is on 

please indicate the nugibgl ■ ^^^^^^^“^^"^ ■ ri p -nradu 'at e in each of these categories. If no 
the quarter system) which are offered to . category mark "0". Please be careful to 

courses are offered to undergraduates in a part ionnth\asls A course which continues for 
define the number of courses involved on a semester-len^ basis. A course n n 
a full year would count as two semester-length courses for PwP®ses of this woie 

excLde conrnes which arc not arailable to nndargradnatas^ g. „ gER HUS. 

BEGINNING OR INTR0D>XT0RY LANGUAGE COURSES: Language courses 
(as opposed to "literature" courses) may involve one or more 
of th'j fbgivbasic skills (listening, speaking, reading, 
writing) diid introduce the student to the basic structure 
and vocabulary of the language. 



INTERMEDIATE LANGUAGE COURSES: Intermediate language courses 
presuppose some prior knowledge of the language and continue 
to stress the further development of one " 
basic skills. Textual materials are introduced chiefly to 
giv e the student practice in handling the language. 



and this box should be checked: C 3QUARTER SYSTEM 



O 



-231 



Appendix C (continued) 



FR SP IT GER RUS 


COMPOSITION COURSES: Courses usu&lly available only to more 
advanced students and which concentrate on the rather inten- 
rfmminnnmnt nf writlnn (see footnote) 












CONVERSATION COURSES: Courses usually available only to 

more advanced students and which concentrate on the develop- 

m°nt Of a Mther high degree of speaking fluency, (see footnote) _ 












PHONETICS COURSES: "Phonetics" courses are distinguished 
from "conversation" courses in that the primary instructional 
emphasis is on the acquisition of a 

and intonation, with somewhat lesser attention paid to the 












ADVANCED GRAMMAR OR SYNTAX COURSES: 






=4 






LITERATURE COURSES: "Literature" courses are devoted primarily 
to the reading of texts in the foreign language (not in trans- 
lation). rather than the study of the language as such. 
Literature courses include the so-called survey courses, as 
well as the study of particular epochs, flenres. schools, or 












SPECIAL READING COURSES: Include here any reading courses 
which are offered for such specifically practica' purposes 
as the acquisition of a reading knowledge in scientific or 
technical fields, include here also any reading courses 
(usually of a terminal nature) which, although they may 
involve more literary readings, aim primarily at the develop- 
ment of basic reading skills rather than the appreciation of 
nnrtinilRi* works involved 












CULTURE AND CIVILIZATION COURSES: 












COURSES IN METHODS OF TEACHING FOREIGN LANGUAGES: 












DIACHRONIC OR HISTORICAL LINGUISTICS COURSES): 












SYNCHRONIC OR DESCRIPTIVE LINGUISTICS (X)URSES: 












Ttxe remaining spaces should be used to describe and number 
aay courses offered to undergraduates by your department 
which are not adequately covered by the above categories: 
















































n — 








... 


— 



COHBINATION COURSES in composition and conversation: ai y?ur — 

composition-conversation coarse, please place numbers or fractional nu^ers J" Joth JJe 
composition and conversation course blanks to reflect the relative emphasis devoted to these 

respective activities. 

Please turn to next page . 



f 
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6 



9) Please check in the table below which of the following activities are part of your foreign 
language program and the extent to which they are required of foreign language majors at some 
time during the undergraduate course of studies. 

YEAR ABROAD: a program providing for an academic year abroad (e.g., Junior year abroad) 

SUMMER ABROAD: a program whereby a student goes to summer school abroad and/or pursues a summer 
study tour abroad. 

LANGUAGE HOUSE; a separate residential unit or section of a dormitory in which a particular 
foreign language is the only language of communication permitted. 

LANGUAGE TABLES: separate tables or cafeteria sections where a particular foreign language is 
the only language of communication, permitted. . . - . 

LANGUAGE CLUBS: extracurricular organizations which promote interest in a particular foreign 
languB ‘^3 and in the culture and history of the countries where the language is spoken. 

PLEDGE: a personal commitment to speak only in the foreign language in designated places and 
at designated times. 

KEY TO TOE TABLE - N.A. (not available) V (voluntary) R (required) 

Please place check marks in aJJL applicable categories and for all applicable languages: 

french Spanish Italian germm bussi 

NA V R NA V R NA V R NA V R NA V 


AN 

R 


YEAR ABROAD: 

SUMMER ABROAD: 

LANGUAGE HOUSE: (during 
(during school year) 

LANGUAGE HOUSE: 

(during the summer) 

LANGUAGE TABLES: 

LANGUAGE CLUBS: 

PLEDGE: 








































































































































































































































































10) Which of the following statements best describes the depart 
the "basic skills" courses (normally, the two-year sequence 
from virtually no knowledge of the language up to a point « 
to handle the language in advanced courses in literature, c 
composition, and so forth.) 

Please check one and fill in the blanks appropriately; 

A. GENERAL. In basic skilLs courses: 

iL 1 Stud©nts 3 r© divid©d* into s©ctions and ©ach s©ctioi 

b) Number of hours per week each section meets: 

2[ ] Some instruction is given to students meeting in 1: 
taught by instructors (not native informanj 

a) Av©rag© numb©r of studonts in larg© s©ctions:__ 
Number of hours per week large sections meet: — 

b) Av©rag© numb©r of studonts in small s©ctions:_ 

Number of hours per week small sections meet: — 

3C ] Some instruction is giv©n to studonts mooting in 1 
sections led by a native informant for purposes of 

a) Average number of students in large sections: — 
Number of hours per week large sections meet; 

b) Average number of students in small sections:^ 
Number of hours per week small sections meet: 

4[ ] Other (describe in terms comparable to the above): 


ment policy in designing and teaching 
( which attempts to bring students 
(here they are sufficiently proficient 
liilture, and civilization, linguistics, 

1 is taught by a single instructor. 

jrge groups and some in small sections 
ts). 

arge groups and some in small 
drill and conversation. 

■■"X 
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B. USE OF LANGUAGE LABORATORY: In conjunction with baaic a.hi.Uf £Our|es: (check one and fill in 

any blanks appropriately) 

l[ ] No language laboratory facilities are available. 

2[ ] Language laboratory facilities are available: 

^hours per week for required work. 

________hours per week at th® student's option. 

C. COURSE EMPHASIS: In basic skills, courses : (check one) 

1[ ] More emphasis is given to listening and speaking than to reading and vnriting. 

2l! ] More emphasis is given to reading and writing than to listening and speaking. 

3[ ] About the same emphasis is given to listening and speaking as to reading and writing. 

4[ ] Our policy may be better described as follows:^ — 



D. RELATIVE AMOUNT OF USE OF ENGLISH:- (Using the following symbols, place one in each of the 
spaces in the appropriate columns of the table) 

F - if the teachers customarily use English 
0 - if the teachers occasionally use English 
R - if the teachers rarely use English 
In basic skills courses . English is used (place one symbol in eacji, space) 



FIRST YEAR 



1) to explain points of gramnar 

2) to explain the meaning of new words and phrases 

3) tc give general instructions to the student 

4) other (specify) — 



c 

c 

i: 

i: 



] 

] 

] 

3 



SECOND YEAR 

C 3 



t 

C 

E 



3 

3 

3 



E ADDITIONAL COMMENTS: Space is provided for a description of any other aspects of departmental 
policy or practice with regard to the basic language courses which Is not covered by the 

above. 



PLEASE HAIL COMPLETED QUESTIONNAIRE (using accompanying reply envelope) to: College 
roreign Language Testing Project, Longfellow Hall. Harvard University, Cambridge, 
Hass. 02138 
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DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE 

OFFICE OF EDUCATION 
WASHINGTON. D.C. 20202 



tCHDRANDUM TO PRESIDENTS OF INSTITUTIONS OF HIGHER LEARNING 

FROM: Kenneth W. Mildfnberger K 

Director^ Division o£ College and University Assistance 



As you know^ the National Defense Education Act o£ 1958 has stimulated considerable 
research and Improvement in the teaching o£ modem foreign languages. 

As a part of Its program of support of research studies in language teaching^ the 
U.S. Office of Education has contracted with Harvard University to administer a new 
.series of foreign language proficiency tests to a representative group of foreign Ian** 
guage majors in their senior year at colleges and universities throughout the Nation. 

1 believe you will agree that a careful study of foreign language programs and students' 
achievements in them will be of great value to all administrators and departments 
concerned. 

The plan calls for testing students in all institutions in the 50 percent sample to be 
studied. The testing program will be offered free of charge to all seniors in the 
institutions who are majoring in any one of the five languages (French^ Spanish^ Italian^ 
German, Russian) for which tests are avdilaule. The tests are to be given early in 1965. 
The data will be used for statistical purposes only without identifying the participating 
individuals or institutions in the report of the survey findings. 

Would you or your academic dean complete the enclosed Institution Information Form and 
send it as soon as possible to Harvard University at the address indicated on the form. 
This form provides space on which you may indicate whether your institution will partici- 
pate; it also asks for certain information which is needed for the study regardless of 
whether your institution participates. If your institution will participate, will you 
appoint a coordinator to receive more detailed correspondence regarding the study and to 
supervise the actual administration of the tests. The coordinator could well be a chair- 
man of a modem foreign language department, a person in charge of language laboratories, 
or a person in charge of a college testing service. 

We feel this is an important project, and we very much hope your institution will be 
able to take part in it. 



Enclosure 
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PJ-251 

Form 2 (revised) 
Nov. 5, 1964 



COLLEGE FOREIGN LANGUAGE TESTING PROJECT 
A Study of the Foreign Language Proficiency of Language Majors 

Near Graduation from College 

Conducted by Harvard University under contract with the U.S. Office of Education 



MEMORANDUM DESCRIBING THE PROJECT 



Under a research contract with the U.S. Office of Education, Harvard University is 
conducting a study in which a new series of foreign language achievement tests will be 
administered without charge to a large sample of foreign language majors graduating from 
United States colleges and universities in the spring of 1965. 

THE TESTS 

The tests to be used in this nationwide project are the Foreign Language Proficiency 
T ests for Teachers and Advanced Students in French, German, Italian, Russian, and Spanish. 
They were produced by the Modern Language Association of America under an NDEA Title VI 
grant, and are designed to measure proficiency in four skill areas: Reading, Writing, 
Listening, and Speaking, and three areas of knowledge relevant to teaching: Civilization 
and Culture, Applied Linguistics, and Professional Preparation. They represent the most 
comprehensive measures currently available at the intermediate and advanced levels of 

foreign language study. ® 

ELIGIBILITY TO TAKE THE TESTS 

The four skills tests will be administered to all senior -class students majoring in 
one of the five languages in which the tests are available. | (Junior class students are 
not eligible.) The three additional subtests related to teaching will be given to students 
certified as completing teacher preparation programs. All students will furthermore be 
administered a short form of a modern language aptitude test as well as a brief questionnaire 
on the history of their study of foreign languages, their interests in foreign language 

study, and other relevant data. 

PURPOSE OF THE TESTING PROGRAM 

The testing program will not only serve a function of national importance but will 
also prove of benefit to both the participating institutions and the students involved. 

On the national level, results of this study will allow a detailed appraisal of the 
functional foreign language proficiency of the pool from which future teachers of foreign 
languages — for all educational levels — will come. For the institutions involved, such 
information will permit an evaluation of their own foreign language programs; and for the 



o 
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students themselves, participation in the study will provide an opportunity to assess 
their own abilities by reference to national norms. 

THE TESTS ARE FREE OF CHARGE 

There will be no charge to the participating institutions or to the shdents for 
the administration of the tests. (The full battery of HLA Foreign Language Proficiency 
Tests normally costs $13.50 per student; the partial battery:, consisting of the four 
skill testSt costs $10.00 per student.) Score reports, together with test norms and 
other interpretative material, will be sent both to the individual students and to the 
appropriate language department chairmen by the early fall of 1965. All information 
collected will be treated as confidential and statistical data will not be identified 
with individual students, departments, or institutions in the published reports. 

DETAILS OF TEST ADMINISTRATION 

With respect to the details of the test administration, the president or academic 
dean of each participating institution has been asked to appoint a coordinator to whom 
the test booklets and o^her materials will be shipped in time for testing in March or 
April, 1965, and who will be directly responsible for the administration of the tests 
and the return of materials. Coordinators and any assisting proctors will receive nominal 
fees for their services. It is expected that the coordinator will be able to work closely 
with each department chairman in securing the greatest possible participation on the part 
of the students. The testing program is expected to have considerable appeal to students 
in view cf the fact that achievement tests of the type to be administered are being 
increasingly used by graduate sdiools, business concerns, and other organizations as 
pari, of their application and placement procedures. Score reports for the different 
skill areas are likely to be useful to students who are contemplating graduate study in 
foreign languages or who plan to use foreign languages in some aspect of their future 
career. 

It is estimated that the total testing time (including rest periods and time for 
instruction) will be close to four hours, except that an additional two hours will be 
required in the case of students taking the parts related to teaching. 

Questions or comments in connection with the project may be referred to the address 
below: 

College Foreign Language Testing Project 
Longfellow Hall 
Harvard University 
Cambridge, Mass. 02138 

(Telephone: Area Code 617, Un 8-7600, Ext. 3458) 
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OE - 4224C 
PJ-251 
Form 20 

COLLEGE FOREIGN LANGUAGE TESTING PROJECT 
A Study of the Foreign Language Proficiency of Language Majors 

near Graduation from College 

INSTITUTION INFORMATION FORM 



Bureau of the Budget No;5l-R454.1 
Approved: November 23, 19M 
Approval expires : December 31, 1965 



Directions : Please print or typewrite throughout. Return this form to: College Foreign Language 

Testing Project, Graduate School of Education, Harvard University, Cambridge, Massachusetts 02138, 

ON OR BEFORE DECEMBER 19, 1964 

1) This item should be answered by all institutions: 

NAME OF INSTITUTION ^ 

WILL YOUR INSTITUTION PARTICIPATE THIS SPRING IN THE TESTING PROJECT? [ ] [ ] 

Yes No 

Institutions not participating : Please answer the next item (item 2) and return this form using 

the reply envelope provided. 

All participating institutions ; Please answer all of the items below and on the reverse side, then 
return this form using the reply envelope provided. 

2) LANGUAGES TAUGHT AT INSTITUTION; NUMBER OF ENROLLED SENIORS 

The tests are given in five languages: French, Spanish, Italian, German, and Russian. Please 
use the boxes below to indicate for which of these languages you have an A.B. (or equivalent) 
degree program, and to report the exact number of seniors who are currently enrolled as "majors" 
or "concentrators" in each language and who will presumably be candidates for the degree at the 
end of the academic year 1964-65. 

FRENCH SPANISH ITALIAN GERMAN RUSSIAN 

BACHELOR * S 

DEGREE [ ] Yes [01 ] [ ] Yes [02] [ ] Yes [04] [ ] Yes [08] [ ] Yes [l6] 

PROGRAM: [ ] No [ ] No [ ] No [ ] No [ ] No 

NUMBER OF SENIORS 

CURRENTLY "MAJORING" 

IN THE LANGUAGE; 



3) DESIGNATION OF COORDINATOR 

The person named below has been appointed coordinator for the testing program and has agreed to 
serve in this capacity: 



Title First name Initial Last name 

4) MAILING ADDRESS OF COORDINATOR 

All tests and other materials will be sent to the coordinator at this address. Please make sure 
that this address is sufficiently complete for mailing purposes. 



TELEPHONE; Area Code No. Extension 



MORE ITEMS ON REVERSE SIDE 
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5) NAMES AND ADDRESSES OF DEPARTMENT CHAIRMEN 






The names and addresses of your foreign language department chairmen are needed for our records, 
not only in order for us to know what departments exist at your institution and which languages they 
cover, but also because it will later be necessary for us to collect certain information directly 
from these chairmen. 

Please use as many as necessary of the boxes below to give the names and mailing addresses of the 
language chairman or chairmen responsible for each of the (test) languages taught at your institution. 
In the brackets at the left of each box, check the language or languages for which each chairman is 
responsible. 

NAME OF CHAIRMAN 



FRENCH [01 ] [ ] 

SPANISH [02] [ J 
ITALIAN [04] [ ] 

GERMAN [08] [ ] 

RUSSIAN [16] [ ] 



Title 



First name 

MAILING ADDRESS 



Initial 



Last name 



NAME OF CHAIRMAN 



FRENCH [01] [ ] 

SPANISH [02] [ ] 

ITALIAN [04] [ ] 

GERMAN [08] [ ] 

RUSSIAN [16] [ ] 



Title 



First name Initial Last name 

MAILING ADDRESS 



NAME OF CHAIRMAN 



FRENCH [01] [ ] 

SPANISH [02] [ ] 

ITALIAN [04] [ ] 

GERMAN [08] [ ] 

RUSSIAN [16] [ ] 



Title 



First name Initial Last name 

MAILING ADDRESS 



NAME OF CHAIRMAN 



FRENCH [01] [ ] 

SPANISH [02] [ ] 

ITALIAN [04] [ ] 

GERMAN [08] [ ] 

RUSSIAN [16] [ ] 



Title 



First name Initial Last name 

MAILING ADDRESS 



NAME OF CHAIRMAN 



FRENCH [01] [ ] 

SPANISH [02] [ ] 

ITALIAN [04] [ ] 

GERMAN [08] [ ] 

RUSSIAN [16] [ ] 



Title 



First name Initial F^ast name 

MAILING ADDRESS 



Deadline for return of form: DECEMBER 19, 1964 (An earlier return is requested if possible) 
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INSTRUCTIONS AND SUGGESTIONS 
TO COORDINATOR FOR 
ADMINISTERING TEST MATERIALS UNDER 
THE COLLEGE FOREIGN LANGUAGE TESTING PROJECT 

SPRING 1965 



This leaflet is intended to provide a master information source for all activljties 
and responsibilities of the institution coordinator in connection with the language^ test- 
ing project. 

CHECK LIST OF MATERIALS 

As a first step, would you please check the materials which you have received in this 

mailing against the list below. 

IN MANILA ENVELOPE LABELED "MATERIALS FOR COORDIl^ATOR"-- 

1) ONE COPY OF THIS LEAFLET 

2) ONE COPY OF "ORDER FORM FOR ADMINISTRATION OF MLA FOREIGN LANGUAGE PROFICIENCY 
TESTS FOR TEACHERS AND ADVANCED STUDENTS UNDER THE COLLEGE FOREIGN LANGUAGE TEST- 
ING PROJECT" 

3) ONE AIR MAIL BUSINESS REPLY ENVELOPE for immediate return of ORDER FORM to 

Harvard ^ „ 

4) ONE "APPLICATION FOR REIMBURSEMENT OF COORDINATORS AND ASSISTING PROCTORS 

5) ONE FIRST CLASS (NON-AIR MAIL) BUSINESS REPLY ENVELOPE for return of APPLICATION 
FOR REIMBURSEMENT form to Harvard following test administration. 

IN MANILA ENVELOPE LABELED "MATERIALS FOR DEPARTMENT CHAIRMEN" — 

1) from 1 to 5 copies of "MEMORANDUM DESCRIBING THE PROJECT" 

2) an equal number of copies of the "QUESTIONNAIRE FOR CHAIRMEN OF FOREIGN LANGUAGE 

DEPARTMENTS" , ^ 

3) an equal number of BUSINEbo REPLY ENVELOPES for return of the QUESTIONNAIRE(s) 

FOR CHAIRMEN, to be returned by the department chairmen to Harvard as soon as 
conveniently possible. 

FREE IN MAILING CARTON (s ) ; 

1) A rufficient supply of the booklet "A DESCRIPTION OF THE MLA FOREIGN LANGUAGE 
PROFICIENCY TESTS FOR TEACHFRS AND ADVANCED STUDENTS" to distribute to each 
foreign language major graduating from your institution in the spring of 1965 in 
one of the five test languages. 

2) An equal number of copies of the leaflet "A LETTER TO COLLEGE SENIORS GRADUATING 
IN THE SPRING OF 1965 WITH DEGREES IN FRENCH, ITALIAN, SPANISH, GERMAN, OR RUS- 
SIAN." These leaflets are enclosed individually in the larger descriptive booklet. 

IF ANY OF THE AEOIE MATERIALS ARE MISSING OR IN INSUFFICIENT SUPPLY, please place a 

collect call to College Foreign Language Testing Project, Area Code 607, UN 8-7600, Ext. 

3459. 

INFORMING STUDENTS ABOUT THE TESTING PROJECT 

As soon as possible, arrangements should be made to distribute one explanatory book- 
let, and the accompanying enclosure, to every senior graduating in the spring of 1965 in 
one of the five test languages. The exact method to be used will vary depending on the 
size and organizational setup of the institution. In some cases, it may be possible for 
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the coordinator to distribute these materials to the students personally. In other cases, 
chairmen of the language departments involved might undertake to make this distribution 

during class periods. 

In any event, arrangements should be made so that all students eligible for testing 
receive these official explanatory materials as soon as possible. 

COMPLETION OF TEST ORDER FORM 

Among other things, the test order form requests information about 

1) the test administration date chosen by your institution 

2) the number of students to be tested in each language. 

With respect to the TEST ADMINISTRATION DATE, each institution is free to choose the 
most convenient testing date, the chief restriction being that this date must fall within 
the period March 1 - May 8, 1965. An additional restriction is explained as follows: 
the testing date chosen for your institution must be at least one month later than the 
date on which the order form is sent to Harvard. Thus, for example, if an institution 
returns the order form on March 15, April 15 is then the earliest testing date which could 
be selected for that institution. Since testing dates in Late April or in May can be ex- 
pected to conflict to some extent with vacation or examination periods at the institution, 
it is recommended that an earlier testing date be chosen if at all possible. 

Regardless of the testing date actually chosen, it is urged that the coordinator make 
every effort to return the test order form to Harvard as soon as he can conveniently do 
so; this will enable the project staff to arrange for test shipments to all institutions 

more efficiently. 

With respect to determining the NUMBER OF STUDENTS to be tested in each language, the 
following procedure is recommended: 

All students have been urged, in the letter accompanying the explanatory booklet, to 
make a commitment to taking the test series on the testing date chosen by the institution. 
It would be well if the coordinator or the department chairmen could arrange to ask each 
student personally if he will participate in the project, and let the student know that 
his answer will affect the number of tests actually ordered, plans made for test adminis- 
tration, and so forth. It is not suggested that the students be formally requi r e d to take 
the tests, but any techniques which might reasonably be used to maximize student partici- 
pation should be in order. 

The MLA proficiency tests are given in two batteries: the Complete Battery (see 

order form) contains the four skill tests— Listening Comprehension, Speaking, Reading, 
Writing, and also the three teaching oriented tests— Applied Linguistics, Civilization and 

Culture, and Professional Preparation. 

The PARTIAL Battery contains the four skill tests only. 

The COMPLETE Battery of 7 tests may be administered only to those senior class stud- 
ents who are enrolled at your institution in a program which is expressly oriented toward 

O 

me 
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the preparation of elementary or secondary school language teachers , and which usually 
involves the completion of a certain series of courses necessary for state certification 
purposes. It will not be possible to administer the complete battery of tests (that is, 
to give the three additional teacher preparation tests) to students who are following reg- 
ular courses of study usually involved in the A.B. degree program in a particular foreign 
language. Further, no tests are to be administered to junior class students. 

The coordinator is asked to determine and enter on the order form the exact number 
of students to be tested in each of the two categories (presumably, this will be the num- 
ber of students who have agreed to participate in answer to the personal request suggested 
earlier) . 

It is important that only the number of tests actually used be ordered. Educational 
Testing Service normally includes a slight overage in test shipments so that such last- 
minute problems as a few additional test takers, a defective booklet, etc. can be handled 
adequately. It is thus not necessary to order additional tests for "safety's sake"; if 
student canvassing has been successfully conducted, the number of tests to be ordered 
should be specifically determinable. 

The additional information required on the order form is largely self-explanatory. 
Please be sure, however, to check the type of recording tape materials which you will re- 
quire for the Listening and Speaking tests; please also complete and sign the "Terms of 
Test Use" on the back of the order form. 

Test order forms should be mailed to Harvard (not to ETS) using the air mail reply 
envelope provided. 

COMMUNICATION WITH DEPARTMENT CHAIRMEN 

The coordinator, in all probability, will already have been in contact with chairmen 
of the language departments for the five test languages (or for as many as are taught at 
the institution), in connection with various testing arrangements. We are asking at this*- 
time that two additional details be conveyed to the department chairmen; 

First, they shoul^be informed that rosters of test scores for all students in their 
departments will be sent to them in the early fall of 1965; along with the rosters will 
come norms and other materials allowing them to appraise their students' performance in 
the various language skills. 

Second, we ask that you send one copy of the "QUESTIONNAIRE FOR CHAIRMEN" to each de- 
partment chairman responsible for one (or more) of the languages in which the tests are 
actually to be given at your institution. According to our records, exactly the right 
number of questionnaires are included in this mailing; if additional questionnaires are 
needed, please inform us to this effect. 

There is no formal deadline for the return of the QUESTIONNAIRE (s) FOR CHAIRMEN to 
Harvard, although we hope that they will be returned within approximately two weeks of re- 
ceipt. Individual business reply envelopes are included with every questionnaire, so that 
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each department chairman nay mail the questionnaire directly to us following completion. 

T EST ADMINISTRATIO N PROCEDURE 

The exact details of test administration will vary from institution to institut on, 
depending on both the total number of students to be tested and the time periods available 
for testing. However, certain general outlines may be suggested. Most studhnts will re- 
quire from 3 1/2 to 4 hours for the testing, divided as follows: 

MLA Foreign Language Proficiency Tests tt 

(Listening Comprehension, Speaking, Reading, Writing; ^ ^ hours 

Approximate total time . . « 

• a. 4 -,* .... 1/2 hour-1 hour 

Student Questionnaire (approximate time) 

y , ^ c \ 1/2 hour 

Modem Language Aptitude Test (short form) 

This testing could occur in one session, either in the morning, the afternoon, or (if 

absolutely necessary) the evening (starting early) . 

Students taking the MLA Proficiency Tests relating to teacher preparation will take 

an additional session of tests lasting about 2 hours. This could be scheduled at any 
convenient time, either on the same day as the other session, or on a different day. 

For the two tests requiring the use of tape recorder equipment (the Listening an 
speaking tests), the. capacity of the language laboratory (if one is used) must be taken 
into account in scheduling the tests. Regardless of the exact laboratory arrangements, 
any given student should be administered the Listening and Speaking tests in that order. 
(The order of administration for the pencll-and-paper tests-including the Modern Language 
Aptitude Test and the student questionnaire-may vary depending on conditions at the in- 

The package of test materials shipped from ETS will Include more specific adminlstr - 
tlon details, including the official testing instructions for each test in the series. 

RETURN OF TESTING MATERIALS 

With the exception of the student questionnaires, which are to be returned to Har- 
vard, all testing materials should be returned to ETS as soon as possible following admin- 
istration, using the prepaid shipping system provided. Student questionnaires are re- 
turned direct to Harvard in special prepaid envelopes which will also be provided. 

uvTMBimcF.MF.MT nv mnpnTMATQR AMD ASSISTING PROCTORS 

Each institution coordinator will receive an honorarium of $30.00 for his services 
in connection with the testing project. Following test administration, the coordinator 
should complete and sign the REIMBURSiEMENT TO COORDINATOR form and return this form to 

Harvard using the reply envelope provided. , 

It is expected that at many institutions, where the total number of students to be 

tested does not exceed about 20, the coordinator will be singly responsible for test 
ministration. At larger Institutions, the coordinator may engage proctors (wHo may be 



o 
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may be either faculty members or graduate students) to assist him; one proctor may be en- 
gaged for each 20 students (or part of 20) over and above the first 20. Thus, for 21-40 
students, one proctor is allowed; for 41-60 students, two proctors may be engaged. Each 
proctor will receive $10.00, and may receive this sum by completing and signing one of the 
"Reimbursement to Assisting Proctor" sections of the reimbursement form. 

ADDITIONAL REMARKS : 

After the test order form has been received for your institution, you will be sent a 
notice giving an INSTITUTION CODE NUMBER for the institution and also a specific range of 
STUDENT CODE NUMBERS to be assigned to the students at the time of testing. It is ex- 
tremely Important that all test materials — including the student questionnaire and the 
Modem Language Aptitude Test sheets — be marked with the correct code numbers. Your at- 
tention to coding details at the time of test administration will help to eliminate pro- 
cessing errors. 

Questions which you may have in regard to the project may be addressed to: 

College Foreign Language Testing Project 
Graduate School of Education 
Harvard University 
Cambridge, Massa'^husetts 02138 

If the situation appears to warrant a telephone call, please do not hesitate to call 
us collect at; Area Code 617, UN 8-7600, Ext. 3459. 

We would like to take this opportunity to thank you for your assistance in the pro- 
ject and express our best wishes for a successful testing session. 
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568 

ORDER FORM 

•for administration of mla foreign language proficiency tests 

FOR TEACHERS AND ADVANCED STUDENTS ^ 

UNDER THE COLLEGE FOREIGN LANGUAGE TESTING PROJECT 



IMPORTANT 

This form must be returned to: 

College Foreign Language Testing Project 
Graduate School of Education 
Harvard University 
Cambridge, Massachusetts 02138 
NO LATER THAN January 25, 1965 



FOR OFFICE USE ONLY 



Rec*d 
Form 
S.M. 
Inst. # 
S.O. Rel'd 
Code Notice Sent 



PLEASE PRINT 



Date of order 
Institution 



Test Administration Date. 
City, State 



Ship to: Name and Title of Coordinator:. 
Institution. 



Street Address 

City, State, and ZIP Code — 

REA Express Address (if different from above )„ 



contains Listening Comprehension*, Speaking*, Reading, 
Writing, Applied Linguistics, Civilization and Culture, 
and Professional Preparation 

FRENCH GERMAN ITALIAN RUSSIAN SPANISH 



COMPLETE BATTERY 

[FOR SENIORS IN TEACHER CER- 
TIFICATION PROGRAMS ONLY] 

Language : 

Number Ordered: ^ ^ .v 

(Please order only the exact number required! 

PARTIAL BATTERY contains Listening Comprehension*, Speaking*, Reading, and Writing 
[FOR SENIORS NOT IN TEACHER 
CERTIFICATION PROGRAMS] 



Language: 
Number Ordered: 



FRENCH 



GERMAN 



ITALIAN RUSSIAN SPANISH 



Language: 

Totals: 



(Please order only the exact number required) 

FRENCH GERMAN ITALIAN RUSSIAN SPANISH 



TOTALS ^ 

(total number of examinees to be tested in each language) 

*Please note that, for the Listening Comprehension and Speaking tests, ETS supplies Master 
Spe wh?Jh contain the stimulus materials. These to 

recorded at 3 3/4 i.p.s., or on a 1200-foot reel recorded at 7 l/2 i.p.s. Be sure to check 
the type desired if either of these tests is used. 

MASTER TAPES for Listening and Speak- [H3 3/4 i.p.s. 600* reel CUT 1/2 i.p.s. 1200 reel 
ing Tests (check type desired) 

♦r.’nr thP Sneakina Test five types of widely used blank tapes (or magnetic discs) are available 
fSi ISe of stutot JLponoeo. suro to oheck the type whioh is used with your re- 

cording equipment. If you have any doubts as to which of these is appropriate for your record 
ing equipment, you should include with the test Order Form a description (including model 
number) of the type of recording equipment to be used. 

RT ANK TAPES for Student re- Ds 3/4 i.p.s. 600* reel D7 l/2 i.p.s. 1200 reel 

s^usoftf sSukU, test ntupe wogLiue (2 spindles) noertrldge (1 sptudle) 

(check type desired) Qmagnetic discs (2 per student) 



Please complete and sign "Terms of Test Use" on reverse side. 



OVER 
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TERMS OF TEST USE 



In consideration of the fulfillment by Educational Testing 
Service of the order stated on the reverse side, the under- 
signed institution (hereinafter called "Test User ) agrees 
as follows: 

Test User will take full responsibility for administering 
the tests and for maintaining the security of the test materials. 

It is understood that the tests will be shown only to the students 
to whom they are administered. 

Educational Testing Service shall ship all test materials via 
REA Express prepaid and the Test User shall return all test 
materials via REA Express collect , unless advance permission is 
obtained from the College Foreign Language Testing Project, 

Harvard University, to have these materials shipped to the institu- 
tion and/or returned to Educational Testing Service by another type 
of carrier. 

All tests and test materials, both used and unused, remain 
the property of the Modern Language Association of America and 
Educational Testing Service. In no event will Test User retain 
them for more than one week after^the scheduled test administration 
date. The used answer sheets and test books, the unused test books, 
and the accessory materials are to be returned to Educational Test- 
ing Service immediately after the test administration. Student 
Questionnaires administered under this program are to be returned 
direct to Harvard University (not to ETS) using the prepaid mailing 
envelopes provided. 

It is understood that test score reporting under this project will 
be undertaken only by Harvard University (not Educational Testing Service) 
and will be restricted :o: 1) a mailing of individual score reports to 
participating students in the early fall of 1965; and 2) a mailing at 
about the same time, of a roster of score reports to the chairmen of 
participating language departments, giving the test scores of students 
in their departments. Beyond these two services, neither Educational 
Testing Service nor Harvard University will undertake to furnish or 
honor requests to furnish at any time transcripts of test scores to 
students, institution officials, or other persons or organizations. 



TEST USER 



Name of Institution 

Authorized Signature of Coordinator 



Title 



Date 
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HARVARD UNIVERSITY 

GRADUATE SCHOOL OF EDUCATION 



College Foreign Language 
Testing Project 



Afpian Way 

Cambridge, Massachusetts 02138 



January 15, 1965 



A LETTER TO COLLEGE SENIORS GRADUATING IN THE SPRING OF 1965 WITH DEGREES 
IN FRENCH, ITALIAN, SPANISH, GERMAN, OR RUSSIAN 

We are sending you this letter (necessarily a form letter) to bring to your attention the 
fact that your school has been asked to participate in a project of foreign language testing to be 
conducted this spring. Under this project, sponsored by the U.S. Office of Education and 
administered here at Harvard, seniors in selected colleges and universities throughout the 
United States are being asked to take a series of four skill tests — Reading, Writing, Listening, 
and Speaking - in the language in which they are majoring (French, Spanish, Italian, German, or 
Russian). In addition, participating students are to be given a short form of a modern language 
aptitude test and also a questionnaire dealing with their language background, prior language 
courses taken, interests in foreign language study, and so forth. 

For students who are certified by their institution as completing teacher preparation 
programs, three additional subtests are given in areas of knowledge particularly relevant to 
teaching; Civilization, and Culture, Applied Linguistics, and Professional Preparation. 

Results of the testing program will provide information about the general level of language 
proficiency of foreign language students throughout the United States near the end of their 
undergraduate training. Research of this type is also of great importance in the planning of more 
effective language teaching sequence.*? at all levels of instruction. 

Our purpose in writing this letter is to urge you strongly to participate in this study, that 
is, to take the series of four skill tests in the language in which you are majoring, the 
language aptitude test and background questionnaire, and, if you are following a teacher- 
preparation program, the three teaching-oriented tests mentioned above. 

We will not attempt to minimize the task which we are requesting of you: participation in the 
program will involve giving approximately four hours*^ of your time on a spring testing 'ate to be 
selected by your school; we realize that this is a considerable contribution on your part. 

We would, however, like to emphasize the benefits to you which will be involved: Firs , test 

results, national norms, and other interpretative materials will be mailed to you as soon 
available (at least by the early fall of 1965). With your scores and other accompanying 
information, you will be able to assess your own ability in each language skill by reference 
to' the achievement of a nationwide sample of language majors. This opportunity to appraise your 
own Strengths (and possibly, weaknesses) in particular areas of foreign language proficiency may be 



"An additional two hours would be required for students taking the teaching-oriented tests 
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of importance to you in making plans for the use of the language in a vocation, in graduate study, 
or for other purposes. Furthermore, you will be able to report the results of these tests to 
graduate schools, business organizations, or others who may request them. 

Second, we would like to point out that the tests are to be gjven free of charge; if you were 
to take the same tests "on your own," you would be required to pay the regular administration fee 
of $10.00 to $13.50, depending on the number of tests taken. 

Third, achievement tests of the type to be administered are used at present and will be 
increasingly used by graduate schools, business concerns, and other organizations as part of their 
application and placement procedures. The opportunity to take tests of this type should be of 
cor "iderable value to you in acquainting you with their general format and content. 

A detailed description of the language tests, together with sample test items, will be found 

in the accompanying booklet, "A Description of the MLA Foreign Language Proficiency Tests for 

Teachers and Advanced Students." Specific questions on the administration of the tests, the 
method of score reporting, and other details are answered at the end of this letter. ^ 

What we would like to request of you at the present time is that you make a commit,jnent to 
participate in the testing program at your school on the date which will be announced in the near 
future. It is of importance to the success of the program that as many students as possible agree 

to participate, and we sincerely hope that you and your fellow language majors will all be willing 

to help us in this undertaking. 



Sincerely yours. 
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ADDITIONAL PROJECT DETAILS 

WHO IS ELIGIBLE TO TAKE THE TESTS IN THIS PROJECT? 

All seniors who are scheduled to graduate this spring from a participating institution and 
who are majoring in one of the languages in which the tests are given (French, Spanish, Italian, 
German, Russian) are eligible to take the four skill tests: Reading, Writing, Listening, Speaking. 
Seniors who are certified as completing a teacher preparation program are eligible to take in 
addition the three teacher-oriented tests: Civilization and Culture, Applied Linguistics, and 
Professional Preparation. 

IS THERE A CHARGE FOR THE TESTS? 

There is no charge either to the student or to the partici -king institution in this special 
testing project. 

ARE THE TESTS DIFFICULT? 

The tests have been deliberately designed to test language achievement at a rather high level 
of proficiency, such as would be expected of a person who has spent several years in language 
study. Thus, undergraduate foreign language majors, students starting graduate study in foreign 
languages, and beginning language teachers would find the tests appropriate. It is expected that 
individual students will find certain tests or parts of these tests quite easy for them, while 
other parts will prove more challenging. One real advantage of this series of tests is the fact 
that all of the important areas of language achievement are involved, allowing the student to 
determine those areas in which additional training or practice may be suggested, as well as those 
areas in which he is proficient. 

HOW WILL TEST SCORES BE REPORTED? 

Early in the fall of 1965, or possibly sooner, a report of scores for all tests taken by the 
student will be mailed to the student at the address specified by him at the time of testing. Along 
with the scores themselves a leaflet with tables of national test norms and explanatory information 
will be provided; these materials will help the student interpret his scores in each language area. 

r 

WHAT OTHER USES WILL BE MADE OF THE TEST SCORES? 

' In the fall of 1965, rosters will be sent to the chairmen of participating language depart- 
ments, giving the test scores of students in their departments. It should be noted, however, 
that student performance on the tests will in no way affect college records. 

The primary purpose for sending a score roster to the institution is to allow the foreign 
language staff to appraise the overall success of their language programs, rather than to examine 
the performance of an individual student as such. 

Beyond the two services described above, individual test scores will not be released to 
students or institutions. It will not be possible to honor requests to furnish additional 
transcripts of test scores either to the student or to other persons or organizations. The student 
is free, of course, to report his own scores, if he so desires, to graduate schools, business 
organizations, an(f so forth, using the official form which will be supplied by this project. 

/ 



OVER, PLEASE 
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✓ 



MAY CERTAIN TESTS BE OMITTED BY THE STUDENT? 

All students who agree to participate in the testing project will be expected to take each of 
the tests in the series — either four or seven tests depending on whether or not the student is 
enrolled in a teacher preparation program. Sufficient testing time will be scheduled by the 
institution, and ample rest periods will be provided. 



WHERE MAY ADDITIONAL INFORMATION BE SECURED? 

The foreign language chairmen at your institution have full information about the testing 
project. A testing coordinator has also been appointed by the institution; it is the 
responsibil^y of the coordinator to supervise the actual administration of the tests. The 
project staff at the Harvard Graduate School of Education will also be glad to receive questions 
or comments, which may be referred to: 



College Foreign Language Testing Project 
Graduate School of Education 
Harvard University 
Cambridge, Massachusetts 02138 



p 
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FJ-251 

Form 11 (revised) 
2-65 



College Foreign Language Testing Project 

DIRECTIONS FOR ADMINISTERING THE QUESTIONNAIRE FOR FOREIGN LANGUAGE MAJORS 

(It is suggested that you read over the questionnaire prior to the actual administra- 
tion, in order to be better prepared to answer any questions that may arise*) 

1) When the students are seated and ready to begin, make sure that they all have ordinary 
lead pencils (with erasers) . Have some extra regular lead pencils on hand to lend to 
the students who may have forgotten to bring their own. 

2) Distribute one questionnaire to each student. 

3) Say: "Note that this is a four-page questionnaire. 

"This questionnaire is a very important part of the project. You may find it 
somewhat lengthy, particularly if you have taken a large number of foreign language 
courses. We have scheduled a full hour during which you may work on this question- 
naire, so there will be ample time for you to answer each item carefully. 

"In answering the questionnaire, you may make use of the list you have brought to 
help you remember the language courses which you have taken. If you finish before 
the hour is up, bring the completed questionnaire to me at the front of the room. 
[Students may be allowed to leave the testing room quietly as they finish.] 
"Directions are printed at the beginning of each part of the questionnaire. 

Please read all directions and answer every item carefully. 

"Are there any questions? [Answer questions.] 

"If you have any later questions, raise your hand and we will help you. 

"Remember to answer all four pages. Please try your best to answer every item. 
"You may begin answering the questionnaire." 

[As the questionnaires are handed in, check them to see that they are complete 
and particularly that the name, address and correct institution and student code 
numbers are present. Please also check that all four pages have been completed.] 

4) NOTES TO THE COORDINATOR: 

In answering pages two and three, some students may have difficulty in determining 
where one "course" ends and another begins. A good working rule in this connection is 
that a new "course" is involved: 

a) when there is a change of teacher, or 

b) whenever there is a definite change in the form of instruction and/or material 
concerned . 

In the rare event that a student has not finished the questionnaire in an hour, 
allow him extra time, if at all possible, on the same day. Be sure to include any late 
questionnaires in the regular shipment back to Harvard. 

It is possible that some students will have taken more than 25 courses in their 
major language, and wi31 require more working space to list and describe the extra 
courses. In this case the students may add 26, 27, etc* below the 25 [on page three] 
or they may use a separate blank sheet of paper. 

RETURN ALL QUESTIONNAIRES TO HARVARD (NOT TO ETS) USING THE PREPAID MAILING ENVELOPE 
PROVIDED. 
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HARVARD UNIVERSITY 

GRADUATE SCHOOL OF EDUCATION 



CoLLEGZ Foreign Language 
Testing Project 



Appian Way 

Cambridge, Massachusetts 02138 
October 5, 1%5 



Memorandum to: The Foreign Language Majors who Took the MLA Foreign Language Proficiency Tests for 

Teachers and Advanced Students in the Spring of 1965 



On the bottom line of the above sticker you will find your Converted Scores on the Modern I.an- 
guage Association Foreign Language Proficiency Tests for Teachers and Advance;’ Students that you 
took in the spring of 1965. The scores are given in the following order: Listening Comprehension, 

Speaking, ’teading. Writing, Applied Linguistics, Citilization and Culture, Professional Preparation. 
The top line of the sticker gives the code number for the institution, your examinee number, the 
language in which you were tested, and your name. 

If no score or other indication appears for n given subtest, our records show that you did not 
take thut subtest. If the symbols AAA, BBB, or CCC appear, there was an irregularity in either the 
administration or the performance of the test such that no valid score could be reported. These 
irregularities occurred particularly in the case of the Speaking test beuause of partial or .otal 
failure of the tape recording. 

The enclosed leaflet will help you in the interpretation of your scores, all of which are given 
in the form of "converted scores" suitable for tse with the norms in the leaflet. I wish to explain 
these norms to you. During the past several years, the Federal Government has supported the holding 
of so-called NDEA (National Defense Education Act) Institutes for teachers of foreign languages, 
generally during the summer months. The students in these Institutes ere for the most part teachers 
of foreign languages in public secondary schools who have applied for admission to the Institutes and 
been accepted. Their travel and living expenses at the Institutes are provided by the government. 
Their previous training in foreign languages and experience in teaching varies widely; some are rel- 
atively recent college graduates, while Jthers are teachers of long experience. The course of study 
at the Institutes includes refresher training in the language, and lectures in applied linguistics, 
civilization and culture and teaching methodology. . 

The students in the NDEA Institutes were tested shortly after arrival ("pretest") and again, 
with an alt'^nate form of the test, at the terminatioh of their study ("posttest"). As the enclosed 
leaflet explains, norms are given for both pretests and posttests. By fallowing the directions on 
the front page of the norms leaflet, you can find the standings of your scores with respect to both 
the pretest and the posttest norms. 

In view of the fact that these MLA proficiency tests (form "A") were administered to you under 
special testing arrangements made for this project only, it will not be possible for either this 
office or Educational Testing Service to honor future requests for score reports. You are free, 
of course, to report your own scores to various schools or other organizations. Students applying 
for state teacher certifica tion should contact their own state education departments to determine 
departmental policy on the use of proficiency test scores for certification purposes. 

Your scores on the Modern Language Aptitude Test, which most of you also took last spring, will 
not be furnished to you since these were collected only for statistical control purposes. 

I wish to thank you, on behalf of the U. S. Office of Education, the Modern Language Associatior 
and Harvard University, for your cooperation in this study, and I hope that the results forwarded 
herewith will be of interest and use to you. 



Sincerely, 

Jbnn B. Carroll 

Director, College Foreign Language 
Testing Project 
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HARVARD UNIVERSITY 

GRADUATE SCHOOL OF EDUCATION 



0 )LLKGE FokEK^N LANGUAGE 

Testing Project 



Appian Wav 

Camdridge, Massachusetts 02138 



October 12, 1965 



Dear Sir: 



Enclosed aic sheets containing the scores of students who were majoring in your department in the 
spring of 1965 and who took the MLA Foreign Language Proficiency Tests for Teachers and Advanced Stu- 
dents at that time. Also enclosed is a leaflet giving norms for these tests as based upon the perfor- 
mance of foreign language teachers attending NDEA Institutes over the past several years. 

As yet, the data from this testing have not been fully analyzed. Eventually, a full report on 
this testing will reach you. In the meantime, we feel that the norms that had been established for 
the teachers who attended NDEA Institutes provide a reasonably satisfactory basis for interpreting 
the performance of the FL majors that were tested at your institution, since it is of interest to com- 
pare the performance of the latter with that of teachers of foreign languages in the public schools. 

Ne do not have exact information on the nature of the sample of teachers that attended NDEA Institutes, 
but there is reason to believe that it is a fairly representative sample of all U. S. teachers of 
foreign languages in public secondary schools. 

The teachers in the NDEA Institutes were tested shortly after arrival ("pretest") and again, with 
an alternate form of the test, at the termination of their study ("posttest"). As the enclosed 
leaflet explains, norms are given for both pretests and posttests. It will probably be of interest to 
you to compare your students' scores both with the pretest and with the posttest norms. 

The identifying information at the top of the rosters gives, first, the three-digit code for your 
institution, followed by the language designation. Information for each student is then listed in the 
following order; student code; student name; test scores for Listening, Speaking, Reading, Writing, 
Applied Linguistics, Civilization and Culture, and Professional Preparation. The scores given are all 
"Converted Scores" appropriate for use with the accompanying norms leaflet. 

If no score or other indication appears for a given subtest, our records show that the student 
did not take that subtest. The symbols AAA, BBB, and CCC are to be interpreted as follows: 



AAA - Invalid Student Speaking Tape 
BBB - Blank Student Speaking Tape 

CCC - Irregularity reported by supervisor (this code is used 
only if it is found that the reported irregularity has 
so affected the student's performance on the test that 
his score cannot be considered valid). 



These scores are also being furnished directly to the students concerned by mail to the addresses 
given by them. It is therefore not necessary for you to take any ‘further action in reporting scores 

to your students. 

I wish to thank you, on behalf of the U. S. Office of Education, the Modern Language Association, 
and Harvard University, for your cooperation in this study, and I hope that the results forwarded 
herewith will be of interest and use. 



Sincerely yours, 



End. 



JBC;jc 
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MLA FOREIGN LANGUAGE PROFICIENCY TESTS FOR TEACHERS AND ADVANCED STUDENTS. 

INTERPRETATIVE LEAFLET AND TEST SCORES 

Pretest Scores Posttest Scores 

L S R W AL C&C PP L S R W AL C&C PP 



NDEA Inst. No. 

The Interpretative Leaflet has been prepared to help you interpret the scores you 
obtained on the MLA Foreign Language Proficiency Tests. 

1. First, you will want to compare your scores for each pretest and correbponding 
posttest. The scores are expressed as converted scores so that pretest and posttest 
score'-, on corresponding tests' are directly comparable. 

2. Next, you will want to compare your scores with the scores of other institute 
participants by determining their percentile ranks from the norms tables. 

The percentile rank for each score shows the percentage of institute participants 
who achieved scores below that score. For example, the percentile rank of a pretest 
score of 52 obtained on the French Listening Comprehension Test is 90. This means that 
a score of 52 on this test is higher than the scores obtained in pretesting by 90 per 
cent of the French institute participants. For another example, the percentxle rank of 
a post test score of 66 obtained on the Russian Writing Test is 75. This means that a 
score of 66 on this test is higher than the scores obtained in posttesting by 75 per cent 
of the Russian institute participants. 

Not all percentile ranks and not all scores are actually shown in the tables. When 
the percentile rank for a particular score is not shown, you may obtain the percentile 
rank through interpolation. For example, a pretest score of 45 obtained on the Spanish 
Reading Test has a percentile rank of 65, and a pretest score of 47 has a percentile rank 
of 70. Since 46 falls one half of the way between the scores listed, the percentile rank 
should be considered to fall one half of the way between the given percentile ranks. In 
other words, the percentile rank of a pretest score of 46 will be 67.5. For another 
example, a posttest score of 105 obtained on the German Speaking Test has a percentile 
rank of 85, and a posttest score of 111 has a percentile rank of 90. Since 107 falls one 
third of the way between the given percentile ranks, the percentile rank of this score 
obtained in posttesting will be 86.7. 

It can be anticipated that a few institute participants will achieve scores that 
have the same percentile rank on both pretest and corresponding posttest. About half of 
the remainder will achieve scores that have higher percentile ranks tor the posttests. 

It is perfectly possible to have a gain in converted ccore from pretest to posttest, and 
still have a percentile rank on the posttest that is lower than the percentile rank on 
the pretest. This is due to the fact that the vast majority of institute participants 
showed a gain in converted scores on post tests. However, the likelihood of showing a 
gain in percentile rank from pretest. to posttest depends in part upon the pretest score. 
In general, participants whose pretest scores fell below the fiftieth percentile are 
likely to have scores that have higher percentile ranks on the corresponding posttests. 



The percentile tables for the French, German, Russian, and Spanish tests are based 
on the test results of all participants studying those languages in the 1961, 1962, and 
1963 institutes. The percentile tables for the Italian tests are based on the test re- 
sults of participants studying Italian in the 1961 and 1962 institutes; these percentile 
tables are labeled ’’tentative" because they are based on so few cases. 

tables for the Professional Preparation Test are based on the results of all participants 
in the 1961, 1962, and 1963 institutes. 

Copyright 1964 by Educational Testing; Service* 

Princeton, New Jersey Berkeley, California 
All rights reserved 

*Reprinted by permission of the Modern Language Association 
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MLA 



FOREIGN LANGUAGE PROFICIENCY TESTS for TEACHERS and ADVANCED STUDENTS 



PRE-TEST 



FRENCH 



POST -TEST 



Percentile 




Ranks 1 


Listening 


99 


56 


97 


55 


95 


54 


90 


52 


85 


50 


80 


48 


75 


46 


70 


43 


65 


41 


60 


39 


55 


38 


50 


36 


45 


35 


40 


34 


35 


33 


30 


32 


25 


31 


20 


30 


15 


29 


10 


28 


5 


26 


3 


25 


1 


22 


No. Cases 


4720 


Mean 


38,1 


Standard 




Deviation 


8.8 



Converted Scores ^ppj civ.and 

Speaking Reading Writing Ling, Culture 
119 67 72 65 67 



111 

106 

98 

93 

89 

85 

83 

80 

77 

74 

71 

68 

66 

63 

61 

58 

55 

51 

47 

43 

40 

36 

4494 

72.1 



65 

63 

59 

56 

54 

51 

49 

47 

45 

44 

42 

40 

39 

37 

36 

35 

33 

32 

30 

28 

26 

24 

4722 

43.3 



68 

66 

61 

58 

55 

53 

50 

48 

46 

44 

42 

40 

38 

36 

35 

33 

31 

29 

26 

24 

22 

20 

4723 

43,0 



62 

59 

56 

54 

52 

50 

49 

48 

47 

46 

44 

43 

42 

41 

40 

39 

38 

36 

35 

32 

31 

29 

4641 

44,9 



62 

59 

55 

53 

51 

49 

47 

46 

45 

44 

43 

42 

41 

40 

39 

38 

36 

35 

33 

31 

30 

27 

4640 

43.6 



19.3 10.8 12,9 8,1 8.5 



Percentile 

Ranks 

99 

97 

95 

90 

85 

80 

75 

70 

65 

60 

55 



50 

45 
40 
35 
30 
25 

20 
15 
10 
5 
3 

1 

No. Cases 

Mean 

Standard 

Deviation 



Listening 
58 

57 
56 
54 
53 
52 

50 
49 
48 
46 
44 

43 

41 
4 ' 

38 
37 
35 

34 
33 
31 
30 
29 



26 

4770 

42.8 

8.6 



GERMAN 



PRE-TEST 



Converted Scores 

Speaking Reading 
122 67 

115 65 

110 64 

104 61 

100 58 

97 56 

93 54 

90 52 

88 50 

86 48 

84 46 

82 45 

79 43 

76 42 

74 40 

71 39 

69 37 

65 36 

62 34 

57 32 

51 30 

48 29 

42 26 

4377 4770 

81.0 45.7 

17.8 10.5 





Appl. 


Civ, and 


Writing Ling, 


Culture 


72 


72 


70 


68 


68 


67 


66 


66 


65 


63 


63 


61 


60 


61 


59 


58 


59 


57 


55 


58 


55 


54 


56 


54 


52 


55 


52 


49 


53 


51 


47 


52 


50 


46 


51 


49 


44 


50 


48 


42 


48 


46 


40 


47 


45 


38 


46 


44 


36 


45 


43 


34 


44 


42 


31 


42 


40 


29 


40 


38 


25 


36 


36 


23 


35 


34 


20 


32 


31 


4769 


4612 


4611 


45.7 


51.1 


49.3 


12.6 


8.9 


8,7 



POST -TEST 



Percentile 

Ranks 


Converted Scores ^ppj 

Listening Speaking Reading Writing Ling. 
56 119 69 ’7 69 


Civ, anc 
Culture 


99 


71 


97 


55 


113 


68 


75 


65 


66 


95 


54 


109 


67 


73 


63 


64 


90 


52 


103 


63 


69 


59 


61 


85 


50 


98 


60 


66 


57 


58 


80 


48 


94 


56 


63 


55 


55 


75 


46 


91 


53 


59 


53 


53 


70 


44 


88 


50 


‘57 


51 


52 


65 


42 


86 


48 


54 


50 


50 


60 


40 


84 


46 


51 


49 


49 


55 


38 


82 


44 


48 


48 


47 


50 


36 


79 


42 


45 


46 


45 


45 


35 


77 


41 


42 


45 


44 


40 


34 


76 


39 


39 


44 


43 


35 


32 


74 


38 


37 


42 


42 


30 


31 


71 


36 


34 


41 


41 


25 




69 


35 


31 


40 


39 


20 


30 


65 


33 


29 


38 


38 


15 


29 


62 


32 


26 


37 


36 


10 


28 


56 


30 


24 


35 


34 


5 


26 


49 


28 


21 


33 


32 


3 


25 


45 


27 


20 


31 


30 


1 


23 


36 


24 


19 


29 


28 


No. Cases 


1102 


1086 


1103 


1103 


1102 


1101 


Mean 


38,4 


79.5 


44.4 


45.8 


46.8 


46.7 


Standard 

Deviation 


9.1 


17.7 


11.9 


16.5 


9.1 


10.0 



Percentile 

Ranks 


Converted Scores Appl. 

Listening Speaking Reading Writing Ling, 


Civ. and 
Culture 


99 


58 


131 


69 


75 


71 


75 


97 


57 


126 


68 


73 


69 


72 


95 


56 


119 


67 


72 


68 


70 


90 


54 


111 


64 


69 


65 


66 


85 


32 


105 


62 


65 


63 


64 


80 


51 


101 


59 


62 


61 


62 


75 


50 


98 


57 


60 


60 


60 


70 


48 


94 


55 


57 


58 


59 


65 


47 


92 


52 


55 


57 


57 


60 


46 


90 


51 


53 


56 


55 


55 


44 


87 


49 


51 


55 


54 


50 


43 


85 


47 


48 


53 


53 


45 


•41 


82 


45 


46 


52 


51 


40 


39 


80 


43 


44 


51 


50 


35 


37 


78 


42 


42 


49 


49 


30 


36 


76 


40 


40 


48 


48 


25 


35 


7*2 


39 


37 


46 


47 


20 


33 


70 


37 


35 


45 


45 


15 


31 


66 


36 


32 


43 


44 


10 


30 


61 


34 


28 


41 


42 


5 


28 


54 


32 


24 


38 


38 


3 


26 


48 




21 


37 


36 


1 


23 


41 


30 


19 


34 


33 


No. Cases 


1256 


1232 


1256 


1258 


1179 


1178 


Mean 


42.1 


85.3 


48.1 


48,3 


53.0 


53.4 


Standard 

Deviation 


9.1 


19.4 


10.9 


14.6 


9.0 


9.5 



- 255 - 



Appendix E- 3 (continued) 



MLA I’OKIilGN l-ANCJUAGli PROFIClIiNCY rESTS for TEACHERS and ADVANCED STUDENTS 



RUSSIAN 

FRE-TEST POST-TEST 



Percentile 




Converted Sc >res 


AppI, 


Civ. and 


Percentile 




Converted Scores 


Appl. 


Civ. and 


Ranks 


Listening 


Speaking Reading Writing Ling. 


Culture 


Ranks 


Listening Speaking Reading Writing Ling. Culture 




55 


122 


68 


78 


59 


69 


99 


57 


122 


68 


79 


62 


72 


97 


53 


111 


63 


76 


56 


67 


V7 


56 


120 


65 


76 


60 


69 


95 


52 


104 


60 


74 


55 


64 


95 


55 


117 


63 


75 


59 


67 


9() 


49 


% 


52 


71 


52 


60 


90 


54 


109 


55 


72 


57 


63 


85 


46 


92 


45 


68 


50 


57 


85 


53 


106 


50 


70 


55 


61 


80 


45 


87 


42 


66 


49 


r5 


80 


51 


102 


47 


68 


53 


59 


75 


43 


84 


39 


63 


48 


54 


75 


50 


99 


43 


66 


52 


58 


70 


42 


81 


37 


60 


46 


52 


70 


49 


96 


41 


64 


51 


56 


65 


41 


79 


36 


57 


45 


51 


65 


47 


94 


39 


62 




55 


60 


39 


76 


35 


55 


44 


49 


60 


46 


92 


37 


59 


50 


54 


55 


38 


74 


34 


52 


43 


48 


55 


44 


89 


36 


57 


49 


53 


50 


37 


72 


33 


49 


42 


47 


50 


43 


87 


35 


55 


48 


51 


45 


36 


70 


32 


46 


41 


45 


45 


41 


84 


34 


53 


46 


50 


40 




67 


31 


44 


40 


44 


40 


40 


83 


33 


50 




49 


35 


35 


65 


30 


40 




43 


35 


39 


81 


32 


47 


45 


48 


30 


34 


62 


29 


37 


39 


41 


30 


37 


78 




45 


44 


47 


25 


33 


61 


28 


33 


38 


40 


25 




76 


31 


42 


43 


46 


20 


32 


58 


27 


31 


37 


39 


20 


36 


72 


30 


37 


41 


45 


15 


31 


55 


26 


29 


36 


38 


15 


35 


69 


29 


35 


40 


44 


10 


30 


52 


25 


26 


34 


36 


10 


34 


64 


27 


32 


38 


42 


5 


28 


47 


23 


23 


32 


33 


5 


33 


58 


25 


28 


36 


39 


3 


27 


43 


22 


21 


31 


31 


3 


32 


55 


24 


27 


35 


37 


1 


24 


35 


20 


20 


28 


29 


1 


29 


49 


23 


25 


34 


34 


No. Cases 


597 


584 


556 


555 


515 


556 


No. Cases 


587 


570 


545 


545 


505 


545 


Mean 


38.4 


73.0 


35.5 


48.6 


42.8 


47.2 


Mean 


43.3 


87.1 


38.2 


53.4 


47.6 


51.9 


Standard 














Standard 














IDeviation 


7.0 


17.7 


10.7 


16.5 


6.8 


9.2 


Deviation 


7.5 


17.0 


10.8 


14.8 


6.7 


8.4 



SPANISH 

PRE-TEST POST-TEST 



Percentile 

Ranks 


Listening 


Converted Scores . 

Appl. 

Speaking Reading Writing Ling. 


Civ. and 
Culture 


Percentile 

Ranks 


Converted Scores » • 

Appl. Civ. and 

Listening Speaking Reading Writing Ling. Culture 


99 


55 


118 


66 


74 


64 


72 


99 


55 


116 


66 


76 


68 


75 


97 


54 


111 


63 


72 


59 


69 


97 


54 


111 


63 


73 


66 


73 


95 


53 


108 


61 


69 


57 


67 


95 


53 


108 


62 


72 


64 


71 


90 


51 


101 


57 


66 


54 


63 


90 


52 


101 


58 


68 


61 


68 


85 


49 


94 


54 


62 


52 


60 


85 


51 


97 


56 


65 


59 


66 


80 


48 


90 


51 


59 


50 


58 


80 


50 


93 


53 


62 


57 


64 


75 


46 


86 


49 


57 


48 


56 


75 


48 


91 


52 


60 


56 


63 


70 


45 


83 


47 


54 


47 


55 


70 


47 


88 


50 


58 


54 


61 


65 


43 


80 


45 


52 


46 


53 


65 


46 


85 


48 


56 


53 


60 


60 


41 


77 


44 


50 


45 


52 


60 


45 


82 


47 


54 


51 


58 


55 


40 


74 


42 


48 


44 


50 


55 


43 


80 


45 


52 


50 


57 


50 


38 


71 


41 


46 


43 


49 


50 


42 


78 


43 


50 


49 


56 


45 


37 


69 


39 


43 


42 


48 


45 


41 


76 


42 


48 


48 


55 


40 


36 


66 


38 


42 


40 


46 


40 


39 


74 


41 


46 


47 


54 


35 


35 


63 


37 


40 


39 


45 


35 


38 


71 


40 


44 


// 


53 


30 


34 


60 


35 


38 




44 


30 


36 


69 


38 


42 


44 


51 


25 


33 


56 


34 


35 


38 


42 


25 


35 


66 


37 


40 


43 


50 


20 


32 


53 


33 


33 


37 


41 


20 


34 


63 


36 


38 


42 


48 


15 


31 


49 


31 


31 


35 


39 


15 


33 


59 


34 


36 


40 


47 


10 


29 


45 


30 


28 


34 


37 


10 


31 


55 


33 


33 


38 


45 


5 


28 


40 


28 


25 


32 


35 


5 


30 


49 


31 


30 


36 


42 


3 


27 


38 


27 


23 


31 


33 


3 


29 


45 


29 


28 


35 


40 


1 


25 


33 


25 


21 


29 


30 


1 


27 


39 


27 


25 


33 


37 


No. Cases 


4394 


4271 


4446 


4444 


4393 


4342 


No. Cases 


4400 


4316 


4544 


4547 


4494 


4443 


Mean 


39.3 


72.1 


42.2 


46.2 


43.3 


49.6 


Mean 


41.8 


78.2 


44.6 


50.2 


49.4 


56.2 


Standard 

Deviation 


8.0 


20.4 


10.2 


13.7 


7.8 


9.7 


Standard 

Deviation 


7.6 


17.5 


9.5 


12.8 


8.5 


8.9 



- 256 - 






Appendix E- 3 (continued) 






MLA FOREIGN LANGUAGE PROFICIENCY TESTS for TEACHERS and ADVANCED STUDENTS 



PRE-TEST 



ITALIAN < tentative) 



POST-TEST 



Percentile I 
Ranks I 


Converted &;ores ^ppj 

Listening Speaking Reading Writing Ling. 


Civ. and 
Culture 


99 


52 


116 


62 


75 


60 


73 


97 




115 


61 


73 


59 


67 


95 


50 


112 


59 


72 


57 


62 


90 


48 


109 


58 


71 


55 


60 


85 


47 


103 


57 


69 


53 


58 


80 


45 


99 


56 


67 


51 


56 


75 


44 


98 


54 


64 


50 


54 


70 


43 


96 


£3 


60 




53 


65 




94 


52 


59 


48 


52 


60 


42 


93 


50 


56 




51 


55 


41 


92 


49 


53 


47 




50 


40 




47 


52 




50 


45 


39 


89 


45 


50 


46 


49 


40 


38 


87 


41 


46 


45 


48 


35 




85 


39 


43 


44 


47 


30 


37 


82 


38 


41 


43 


46 


25 


36 




36 


38 


41 


44 


20 


35 


78 


33 


37 




43 


15 


34 


70 




33 


39 


42 


10 


32 


65 


32 


31 


37 


39 


5 


30 


61 


29 


29 


35 


37 


3 


28 


55 


27 


27 


33 


35 


1 


26 


46 


25 


24 


29 


33 


No. Cases 


64 


62 


64 


64 


64 


64 


Mean 


40.0 


88.2 


45.2 


51.1 


46.1 


49.8 


Sundard 

Deviation 


6.0 


15.4 


10.3 


14.4 


6.5 


8.0 



Percentile 




Converted Scores 




Appl. 


Ranks 


Listening Speaking Readirg Writing Ling, 


99 


51 


126 


64 


80 


66 


97 


49 


123 




78 


64 


95 


48 


121 


63 


77 


63 


90 




114 




74 


61 


85 


47 


113 


60 


72 


60 


80 


46 


111 


59 


71 


58 


75 


45 


110 


57 


69 


56 


70 


44 


107 


54 


66 


55 


65 




105 


52 


62 


54 


60 


43 


104 


50 


61 




55 


42 


102 




60 


53 


50 




100 


47 


57 


51 


45 


40 


98 


46 


52 


50 


40 


39 


97 


45 


51 


48 


35 


38 


94 


44 


49 


46 


30 


37 


91 


41 


47 




25 


36 


88 


40 


42 


44 


20 


35 


86 


38 


38 


43 


15 


33 


79 


34 


37 


42 


10 


32 


75 


33 


35 


41 


5 


30 


72 


29 


33 


37 


3 


28 


68 


28 


30 


36 


1 


27 


65 


25 


26 


34 


No. Cases 


64 


62 


64 


64 


64 


Mean 


40.5 


98.0 


47.5 


55.3 


50.7 


Standard 












"Teviation 


5.9 


14.8 


10.7 


14.7 


7.8 



PROFESSIONAL PR PARATION 
ALL LANGUAGES 





PRE-TEST 


Percentile 


Ranks 


Converted Scores 


99 


76 


97 


74 


95 


72 


90 


70 


85 


68 


80 


67 


75 


65 


70 


64 


65 


63 


60 


62 


55 


61 


50 


60 


45 


59 


40 


58 


35 


57 


30 


55 


25 


54 


20 


52 


15 


51 


10 


49 


5 


46 


3 


44 


1 


39 


No. Cases 


10771 


Mean 


59.5 


Standard 




Deviation 


8.2 



Percentile 


POST-TEST 


Ranks 


Converted Scores 


99 


79 


97 


77 


95 


76 


90 


75 


85 


74 


80 


72 


75 




70 


71 


65 


70 


60 

55 


69 


50 


68 


45 


67 


40 


66 


35 


65 


30 


64 


25 


63 


20 


62 


15 


60 


10 


58 


5 


55 


3 


53 


1 


49 


No. Cases 


10909 


Mean 


67.0 


Standard 


' 


Deviation 


6.5 



——Reprinted by permission of the Modem Language Association 




Civ. and 
Culture 
76 

71 

67 

65 

64 

62 

61 

60 

58 

57 

56 

54 

53 

52 

51 

50 

49 

47 

46 

43 

40 

38 

36 

64 

54.5 

8.5 



